-- Logs begin at Wed 2022-08-17 22:25:39 UTC, end at Thu 2022-08-18 00:03:05 UTC. -- Aug 17 23:29:28.460533 dv-cinder-820027-20-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 17 23:29:28.467941 dv-cinder-820027-20-iscsi devstack@keystone.service[85821]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 17 23:29:28.470902 dv-cinder-820027-20-iscsi devstack@keystone.service[85821]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 17 23:29:28 2022] *** Aug 17 23:29:28.470902 dv-cinder-820027-20-iscsi devstack@keystone.service[85821]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 17 23:29:28.470902 dv-cinder-820027-20-iscsi devstack@keystone.service[85821]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 17 23:29:28.470902 dv-cinder-820027-20-iscsi devstack@keystone.service[85821]: nodename: dv-cinder-820027-20-iscsi Aug 17 23:29:28.470902 dv-cinder-820027-20-iscsi devstack@keystone.service[85821]: machine: x86_64 Aug 17 23:29:28.470902 dv-cinder-820027-20-iscsi devstack@keystone.service[85821]: clock source: unix Aug 17 23:29:28.470902 dv-cinder-820027-20-iscsi devstack@keystone.service[85821]: pcre jit disabled Aug 17 23:29:28.470902 dv-cinder-820027-20-iscsi devstack@keystone.service[85821]: detected number of CPU cores: 4 Aug 17 23:29:28.470902 dv-cinder-820027-20-iscsi devstack@keystone.service[85821]: current working directory: / Aug 17 23:29:28.470902 dv-cinder-820027-20-iscsi devstack@keystone.service[85821]: detected binary path: /usr/bin/uwsgi-core Aug 17 23:29:28.470902 dv-cinder-820027-20-iscsi devstack@keystone.service[85821]: your processes number limit is 47680 Aug 17 23:29:28.470902 dv-cinder-820027-20-iscsi devstack@keystone.service[85821]: your memory page size is 4096 bytes Aug 17 23:29:28.471539 dv-cinder-820027-20-iscsi devstack@keystone.service[85821]: detected max file descriptor number: 2048 Aug 17 23:29:28.471539 dv-cinder-820027-20-iscsi devstack@keystone.service[85821]: lock engine: pthread robust mutexes Aug 17 23:29:28.471539 dv-cinder-820027-20-iscsi devstack@keystone.service[85821]: thunder lock: enabled Aug 17 23:29:28.471539 dv-cinder-820027-20-iscsi devstack@keystone.service[85821]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 17 23:29:28.471539 dv-cinder-820027-20-iscsi devstack@keystone.service[85821]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 17 23:29:28.512919 dv-cinder-820027-20-iscsi devstack@keystone.service[85821]: Python main interpreter initialized at 0x5639733289c0 Aug 17 23:29:28.512919 dv-cinder-820027-20-iscsi devstack@keystone.service[85821]: python threads support enabled Aug 17 23:29:28.512919 dv-cinder-820027-20-iscsi devstack@keystone.service[85821]: your server socket listen backlog is limited to 100 connections Aug 17 23:29:28.512919 dv-cinder-820027-20-iscsi devstack@keystone.service[85821]: your mercy for graceful operations on workers is 90 seconds Aug 17 23:29:28.513281 dv-cinder-820027-20-iscsi devstack@keystone.service[85821]: mapped 403077 bytes (393 KB) for 2 cores Aug 17 23:29:28.513355 dv-cinder-820027-20-iscsi devstack@keystone.service[85821]: *** Operational MODE: preforking *** Aug 17 23:29:28.513598 dv-cinder-820027-20-iscsi devstack@keystone.service[85821]: *** uWSGI is running in multiple interpreter mode *** Aug 17 23:29:28.513598 dv-cinder-820027-20-iscsi devstack@keystone.service[85821]: spawned uWSGI master process (pid: 85821) Aug 17 23:29:28.513803 dv-cinder-820027-20-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 17 23:29:28.515417 dv-cinder-820027-20-iscsi devstack@keystone.service[85821]: spawned uWSGI worker 1 (pid: 85834, cores: 1) Aug 17 23:29:28.515417 dv-cinder-820027-20-iscsi devstack@keystone.service[85821]: spawned uWSGI worker 2 (pid: 85835, cores: 1) Aug 17 23:29:28.515941 dv-cinder-820027-20-iscsi devstack@keystone.service[85821]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 17 23:29:32.793000 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 17 23:29:32.793000 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: from cryptography.utils import int_from_bytes Aug 17 23:29:32.793779 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 17 23:29:32.793779 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: from cryptography.utils import int_from_bytes Aug 17 23:29:32.794185 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 17 23:29:32.794185 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: from cryptography.utils import int_from_bytes Aug 17 23:29:32.794294 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 17 23:29:32.794294 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: from cryptography.utils import int_from_bytes Aug 17 23:29:32.843975 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 17 23:29:32.844142 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 17 23:29:32.844220 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 17 23:29:32.844311 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 17 23:29:32.844383 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] command line args: [] {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 17 23:29:32.844457 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] command line args: [] {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 17 23:29:32.844636 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 17 23:29:32.844721 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 17 23:29:32.844807 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ================================================================================ {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 17 23:29:32.844883 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ================================================================================ {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 17 23:29:32.845059 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] admin_token = **** {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.845136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] admin_token = **** {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.845307 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] config_dir = [] {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.845396 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] config_dir = [] {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.845496 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.845573 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.845679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] config_source = [] {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.845762 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] config_source = [] {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.845860 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] debug = True {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.845963 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] debug = True {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.846088 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.847279 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.847279 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.847433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.847499 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] insecure_debug = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.847557 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] insecure_debug = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.847618 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.847675 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.847675 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.847790 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.847853 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] list_limit = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.847910 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] list_limit = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.847994 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] log_config_append = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.848069 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] log_config_append = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.848133 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.848198 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.848261 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] log_dir = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.848323 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] log_dir = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.848385 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] log_file = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.848446 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] log_file = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.848508 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.848573 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.848631 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.848687 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.848742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.848797 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.848953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.849280 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.849405 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.849489 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.849566 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.849682 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.849764 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.849844 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.849957 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.850045 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.850128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.850199 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.850272 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.850348 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.850459 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.850537 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.850607 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.850718 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.850833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.850916 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.851079 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] notification_format = cadf {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.851160 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] notification_format = cadf {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.851354 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.851432 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.851845 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.851989 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.852065 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] publish_errors = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.852189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] publish_errors = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.852265 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.852382 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.852502 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.852584 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.852665 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.852734 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.852919 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.853057 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.853160 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.853237 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.853397 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] standard_threads = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.853479 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] standard_threads = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.853632 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] strict_password_check = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.853708 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] strict_password_check = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.853875 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.853957 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.854055 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] use_eventlog = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.854130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] use_eventlog = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.854231 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] use_journal = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.854306 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] use_journal = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.854406 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] use_json = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.854482 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] use_json = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.854581 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] use_stderr = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.854661 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] use_stderr = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.854756 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] use_syslog = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.854837 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] use_syslog = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.854988 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] watch_log_file = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.855064 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] watch_log_file = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 23:29:32.855246 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.855320 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.855486 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.855562 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.855748 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.855825 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.856008 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.856088 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.856264 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.856347 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.856505 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.856581 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.856753 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.856828 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.856991 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.857073 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.857240 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.857318 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.857488 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.857564 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.857737 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.857858 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.857981 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.858652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.858773 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.858884 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.859005 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.859120 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.859239 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.859358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.859470 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.859581 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.859692 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.859807 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.859927 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.860065 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.860178 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.860294 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.860432 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.860549 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.860668 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.860781 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.860918 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.861040 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.861183 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.861314 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.861427 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.861537 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.861679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.861795 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.861905 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.862007 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.862139 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.862247 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.862374 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.862482 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.862614 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.862724 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.862865 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.862974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.863117 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.863238 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.863350 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.863530 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.863644 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.863754 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.863863 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.864028 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.864153 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.864275 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.864387 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.864487 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.864596 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.864712 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.864829 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.864949 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.865061 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.865170 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.865283 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.865393 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.865508 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.865628 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.865745 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.865856 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.865965 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.866075 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.866190 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.866310 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.866421 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.866531 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.866641 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.866760 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.866871 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.866989 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.867100 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.867216 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.867331 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.867442 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.867558 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.867677 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.867789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.867898 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.868022 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.868145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.868262 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.868381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.868494 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.868636 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.868757 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.868870 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.868989 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.869101 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.869223 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.869337 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.869448 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.869565 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.869688 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.869800 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.869911 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.870022 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.870132 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.870242 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.870360 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.870473 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.870585 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.870703 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.870824 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.870945 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.871059 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.871178 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.871299 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.871412 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.871522 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.871634 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.871745 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.871861 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.872000 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.872115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.872227 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.872338 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.872448 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.872565 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.872687 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.872796 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.872907 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.873019 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.873129 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.873240 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.873359 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.873472 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.873589 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] cache.enabled = True {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.873800 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] cache.enabled = True {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.874446 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.874561 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.874680 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.874800 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.874912 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.875023 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.875140 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.875252 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.875439 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.875568 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.875568 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.875786 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.875897 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.876046 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.876172 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.876286 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.876397 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.876508 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.876619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.876729 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.876850 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.876963 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.877075 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.877185 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.877302 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.877420 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.877546 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.877667 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.877781 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.877893 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.878003 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.878114 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.878233 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.878345 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.878456 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.878566 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.878685 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.878814 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.878928 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.879048 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.879161 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.879273 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.879384 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.879494 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.879605 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.879723 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.879836 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.879949 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.880106 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.880219 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.880341 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.880462 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.880575 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.880686 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.880798 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.880914 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.881028 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] auth.external = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.881148 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] auth.external = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.881261 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] auth.mapped = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.881372 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] auth.mapped = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.881482 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.881625 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.881749 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.881864 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.881976 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] auth.password = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.882096 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] auth.password = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.882210 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] auth.token = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.882323 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] auth.token = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.882440 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.882552 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.882665 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] catalog.caching = True {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.882785 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] catalog.caching = True {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.882899 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.883011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.883125 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.883236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.883347 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.883467 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.883582 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.883694 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.883805 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.883917 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.884053 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] credential.caching = True {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.884179 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] credential.caching = True {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.884293 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] credential.driver = sql {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.884405 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] credential.driver = sql {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.884524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.884638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.884751 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.884870 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.884983 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.885095 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.885208 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.885319 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.885430 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.885549 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.885662 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.885774 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.885885 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.885996 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.886107 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.886227 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.886342 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.886454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.886565 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.886676 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.886788 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.886907 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.887021 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.887139 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.887252 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.887371 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.887487 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.887608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.887722 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] federation.caching = True {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.887834 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] federation.caching = True {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.887945 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.888073 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.888185 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] federation.driver = sql {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.888309 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] federation.driver = sql {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.888424 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.888536 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.888648 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.888759 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.888869 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.889053 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.889178 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.889325 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.889443 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.889556 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.889676 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.889790 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.889902 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.890014 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.890126 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.890238 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.890358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.890473 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.890591 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] identity.caching = True {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.890703 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] identity.caching = True {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.890815 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.890927 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.891055 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.891170 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.891282 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.891399 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.891519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.891632 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.891753 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] identity.driver = sql {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.891874 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] identity.driver = sql {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.892040 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.892156 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.892268 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.892380 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.892503 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.892627 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.892741 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.892859 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.892973 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.893085 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.893205 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.893318 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.893429 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.893540 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.893653 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.893764 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.893884 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.893998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.894110 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.894221 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.894333 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.894445 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.894565 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.894680 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.894792 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.894903 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.895015 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.895127 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.895250 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.895365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.895477 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.895588 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.895701 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.895818 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.895939 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.896070 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.896200 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.896314 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.896426 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.896537 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.896660 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.896775 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.896888 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.897011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.897126 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.897238 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.897358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.897475 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.897587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.897699 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.897811 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.897922 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.898045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.898159 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.898271 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.898382 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.898493 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.898612 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.898726 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.898840 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.898952 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.password = **** {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.899063 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.password = **** {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.899174 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.899294 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.899407 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.899519 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.899631 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.899743 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.899855 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.899993 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.900109 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.900221 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.900343 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.900456 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.900568 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.900688 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.900809 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.900924 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.901035 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.901147 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.901258 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.901378 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.901492 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.901604 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.901715 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.901832 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.901945 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.902066 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.902180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.902291 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.902403 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.user = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.902514 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.user = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.902625 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.902997 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.903112 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.903225 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.903337 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.903449 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.903561 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.903680 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.903794 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.903906 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.904093 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.904206 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.904318 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.904441 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.904556 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.904669 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.904780 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.905423 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.905544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.905665 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.905779 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.905892 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.906004 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.906115 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.906228 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.906348 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.906462 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.906574 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.906685 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.906796 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.906907 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.907027 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.907141 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.907254 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.907365 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.907476 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.907587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.907707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.907820 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.907932 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.908069 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.908182 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.908295 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.908419 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.908533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.908646 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.908764 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.908877 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.908988 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.909109 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.909223 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.909336 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.909448 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] policy.driver = sql {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.909559 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] policy.driver = sql {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.909670 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.909790 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.909904 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.910015 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.910127 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.910238 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.910351 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] receipt.caching = True {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.910471 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] receipt.caching = True {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.910585 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.910698 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.910809 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.910921 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.911032 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.911153 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.911268 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.911381 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.911493 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.911605 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.911717 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] resource.caching = True {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.911902 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] resource.caching = True {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.912031 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.912145 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.912256 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] resource.driver = sql {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.912368 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] resource.driver = sql {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.912480 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.912604 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.912718 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.912830 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.912943 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.913054 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.913166 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] revoke.caching = True {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.913285 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] revoke.caching = True {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.913399 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.913513 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.913625 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.913736 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.913848 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] role.cache_time = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.913967 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] role.cache_time = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.914081 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] role.caching = True {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.914192 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] role.caching = True {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.914304 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] role.driver = sql {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.914415 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] role.driver = sql {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.914535 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] role.list_limit = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.914649 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] role.list_limit = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.914761 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.914873 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.914985 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.915097 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.915218 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.915332 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.915446 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.915557 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.915671 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.915783 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.915904 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.916075 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.916188 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.916300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.916411 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.916523 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.916648 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.916763 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.916875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.916987 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.917099 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.917211 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.917332 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.917446 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.917558 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.917670 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.917869 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.918064 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.918193 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.918314 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.918428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.918540 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.918653 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.918767 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.918887 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.919023 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.919140 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.919282 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.919399 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.919519 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.919634 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.919747 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.919860 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.920023 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.920141 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.920266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.920382 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.920495 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.921178 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.921298 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.921409 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.921531 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.921645 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.921757 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.921868 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.921980 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.922092 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.922212 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.922325 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.922437 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.922549 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.922660 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.922772 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] token.cache_time = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.922892 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] token.cache_time = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.923007 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] token.caching = True {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.923178 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] token.caching = True {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.923294 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.923405 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.923516 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] token.provider = fernet {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.923637 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] token.provider = fernet {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.923752 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.923864 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.923999 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.924114 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.924228 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.924352 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.924467 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.924579 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.924691 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.924803 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.924914 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.925039 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.925154 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] trust.driver = sql {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.925267 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] trust.driver = sql {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.925379 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.925491 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.925603 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.925723 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.925837 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.925950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.926061 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.926173 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.926285 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.926406 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.926519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.926632 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.926743 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.926855 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.926966 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.927134 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.927254 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] database.connection = **** {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.927402 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] database.connection = **** {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.927533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.927655 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.927769 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.927882 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.928011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.928148 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.928279 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.928397 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.928510 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.928621 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.928733 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.928844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.928964 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.929078 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.929190 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.929303 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.929414 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.929526 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.929646 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.929760 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.929872 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.929984 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.930095 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.930229 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.930355 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.930469 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.930580 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.930692 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.930803 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.930915 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.931096 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.931212 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.931326 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.931439 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.931550 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.931661 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=85835) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 17 23:29:32.931783 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 23:29:32.931783 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=85834) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 17 23:29:32.960445 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=85835) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:32.960639 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=85834) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:32.960726 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=85835) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:32.960873 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=85834) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:32.960955 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=85835) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:32.961101 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=85834) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:32.969982 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=85834) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:32.970308 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=85835) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:32.998236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=85834) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:32.998466 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=85834) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:32.998745 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=85835) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:32.998978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=85835) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.051866 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=85834) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.052263 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=85835) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.100673 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=85834) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.100808 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=85835) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.100941 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=85834) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.101020 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=85835) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.256833 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=85834) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.257041 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=85835) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.257125 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=85834) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.257199 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=85835) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.257272 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=85834) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.257346 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=85835) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.257477 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=85834) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.257560 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=85835) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.257695 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=85834) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.257777 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=85835) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.257909 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=85834) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.257998 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=85835) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.281761 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=85835) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.281992 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=85835) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.282109 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=85834) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.282219 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=85835) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.282343 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=85834) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.282437 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=85835) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.282570 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=85834) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.282651 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=85835) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.282794 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=85834) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.282874 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=85835) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.283014 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=85834) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.283094 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=85835) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.283231 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=85834) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.283447 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=85834) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.293986 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=85835) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.294248 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=85834) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.306685 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=85835) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.306873 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=85835) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.306956 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=85834) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.307101 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=85835) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.307193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=85834) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.307324 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=85834) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.307404 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=85835) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.307543 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=85834) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.307624 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=85835) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.307757 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=85835) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.307846 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=85834) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.307985 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=85834) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.308068 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=85835) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.308207 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=85834) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 23:29:33.324598 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.324808 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 17 23:29:33.325124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.325204 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.325335 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 17 23:29:33.325416 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 17 23:29:33.325612 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.325699 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.325869 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 17 23:29:33.325949 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.326149 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.326382 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.326469 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.326651 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.327437 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.327705 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.336525 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.336807 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.337058 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.337141 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.337339 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.337428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.337612 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.337896 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.344883 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=85835) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 23:29:33.345414 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=85834) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 23:29:33.347755 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=85835) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 23:29:33.348072 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.348314 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=85834) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 23:29:33.348395 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.348605 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.348686 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.348893 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.348981 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.349158 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.349238 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.349434 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.349514 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.349702 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.349790 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.349976 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.350088 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.350255 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.350369 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.350640 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.350721 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.350921 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.351194 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.373115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.373695 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.374595 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=85835) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 23:29:33.374884 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.375146 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=85834) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 23:29:33.375432 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.379544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=85835) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 23:29:33.379834 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.380120 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=85834) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 23:29:33.380207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.380408 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.380694 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.386459 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=85835) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 23:29:33.386751 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.387048 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=85834) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 23:29:33.387335 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.391358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=85835) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 23:29:33.391649 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.391936 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.392043 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=85834) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 23:29:33.392246 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.392328 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.392527 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.392618 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.392805 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.392885 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.393080 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.393159 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.393350 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.393439 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.393622 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.393943 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.411188 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.411471 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.411747 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.412058 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.415333 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=85835) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 23:29:33.415926 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=85834) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 23:29:33.418594 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.418895 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.419155 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.419456 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.423160 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=85835) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 23:29:33.423707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=85834) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 23:29:33.426325 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=85835) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 23:29:33.426861 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=85834) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 23:29:33.429599 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.429904 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.430110 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.430203 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.430414 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.430510 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.430706 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.430787 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.430996 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.431078 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.431278 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.431561 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.446977 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=85835) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 23:29:33.447279 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.447467 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=85834) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 23:29:33.447581 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.447765 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.447874 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.448062 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.448333 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.455244 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.455735 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.456805 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.457090 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.457285 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.457567 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.459740 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=85835) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 23:29:33.460058 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.460233 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=85834) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 23:29:33.460331 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.460522 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.460642 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.460811 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.460926 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.461115 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.461398 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.472955 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=85835) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 23:29:33.473245 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.473329 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=85834) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 23:29:33.473528 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.473616 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.473811 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.473911 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.474090 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.474182 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.474369 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.474452 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.474647 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.474730 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.474987 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.490115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=85835) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 23:29:33.490399 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=85834) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 23:29:33.493185 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=85835) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 23:29:33.493475 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=85834) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 23:29:33.496098 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.496388 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.497532 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.497840 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.499025 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=85835) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 23:29:33.499313 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.499398 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=85834) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 23:29:33.499614 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.499697 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.499914 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.506102 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.506389 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.507545 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=85835) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 23:29:33.507841 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=85834) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 23:29:33.510454 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.510750 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.510844 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.511040 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.511120 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.511322 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.511413 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.511597 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.519888 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=85835) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 23:29:33.520087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=85834) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 23:29:33.520214 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.520380 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.520498 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.520664 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.527351 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=85835) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 23:29:33.527539 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=85834) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 23:29:33.527624 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.528099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.528200 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.528388 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.528471 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.528667 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.528751 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.528943 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.529037 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.529228 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.529311 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.529493 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.529573 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.529764 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.529846 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.530102 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.530190 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.530382 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.530472 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.530645 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.530796 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.530949 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.531118 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.531292 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=85835) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.531378 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.531455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=85834) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 23:29:33.566112 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5639733289c0 pid: 85835 (default app) Aug 17 23:29:33.566252 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5639733289c0 pid: 85834 (default app) Aug 17 23:29:33.569482 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec9708-a67a-45cb-ad25-c15a3bc98748 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:33.569770 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec9708-a67a-45cb-ad25-c15a3bc98748 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:33.570044 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec9708-a67a-45cb-ad25-c15a3bc98748 None None] PATH_INFO: `/v3` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:33.574187 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1/1] 10.222.0.26 () {50 vars in 791 bytes} [Wed Aug 17 23:29:33 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:29:40.064199 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131d9da-9185-4ea3-8c34-662bb1f0f2d3 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:40.064706 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131d9da-9185-4ea3-8c34-662bb1f0f2d3 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:40.064706 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131d9da-9185-4ea3-8c34-662bb1f0f2d3 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:40.068995 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1/2] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:29:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 23:29:40.074038 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bee49-4fc7-4644-99ae-a9714a8aec3e None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:40.074455 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bee49-4fc7-4644-99ae-a9714a8aec3e None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:40.074575 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bee49-4fc7-4644-99ae-a9714a8aec3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:40.144741 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: INFO dbcounter [None req-3c8bee49-4fc7-4644-99ae-a9714a8aec3e None None] Registered counter for database keystone Aug 17 23:29:40.164157 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG oslo_db.sqlalchemy.engines [None req-3c8bee49-4fc7-4644-99ae-a9714a8aec3e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=85835) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 17 23:29:40.170197 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writer thread running {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 17 23:29:40.347271 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG passlib.handlers.bcrypt [None req-3c8bee49-4fc7-4644-99ae-a9714a8aec3e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=85835) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 17 23:29:40.347615 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG passlib.handlers.bcrypt [None req-3c8bee49-4fc7-4644-99ae-a9714a8aec3e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=85835) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 17 23:29:40.401768 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-3c8bee49-4fc7-4644-99ae-a9714a8aec3e None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:29:40.403615 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-3c8bee49-4fc7-4644-99ae-a9714a8aec3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:29:40.531645 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 2/3] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:29:40 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 458 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 17 23:29:40.998340 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131d9da-9185-4ea3-8c34-662bb1f0f2d3 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:29:41.066729 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: INFO dbcounter [None req-8131d9da-9185-4ea3-8c34-662bb1f0f2d3 None None] Registered counter for database keystone Aug 17 23:29:41.087730 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG oslo_db.sqlalchemy.engines [None req-8131d9da-9185-4ea3-8c34-662bb1f0f2d3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=85834) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 17 23:29:41.093892 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writer thread running {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 17 23:29:41.211496 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131d9da-9185-4ea3-8c34-662bb1f0f2d3 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:29:41.215738 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f98ab1-f8c4-4eb7-8920-f9c5bee9869f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:29:41.216387 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f98ab1-f8c4-4eb7-8920-f9c5bee9869f admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:41.216667 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f98ab1-f8c4-4eb7-8920-f9c5bee9869f admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:41.216937 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f98ab1-f8c4-4eb7-8920-f9c5bee9869f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:41.225458 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f98ab1-f8c4-4eb7-8920-f9c5bee9869f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:29:41.276553 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b4f98ab1-f8c4-4eb7-8920-f9c5bee9869f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 17 23:29:41.276842 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f98ab1-f8c4-4eb7-8920-f9c5bee9869f admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:29:41.286961 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 2/4] 10.222.0.26 () {58 vars in 1324 bytes} [Wed Aug 17 23:29:40 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 23:29:41.292919 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8bee49-4fc7-4644-99ae-a9714a8aec3e None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:29:41.307868 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8bee49-4fc7-4644-99ae-a9714a8aec3e None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:29:41.312386 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49597fc3-50f6-48f0-a541-8b161b3f3b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:29:41.313478 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49597fc3-50f6-48f0-a541-8b161b3f3b0a admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:41.313763 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49597fc3-50f6-48f0-a541-8b161b3f3b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:41.314061 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49597fc3-50f6-48f0-a541-8b161b3f3b0a admin admin] PATH_INFO: `/v3/projects/fd475051e9cd456abbb6cff649b374cc` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:41.315631 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49597fc3-50f6-48f0-a541-8b161b3f3b0a admin admin] RBAC: Authorizing `identity:get_project(project_id=fd475051e9cd456abbb6cff649b374cc)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:29:41.367971 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: WARNING keystone.common.rbac_enforcer.enforcer [None req-49597fc3-50f6-48f0-a541-8b161b3f3b0a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 17 23:29:41.368333 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49597fc3-50f6-48f0-a541-8b161b3f3b0a admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:29:41.372793 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 3/5] 10.222.0.26 () {56 vars in 1207 bytes} [Wed Aug 17 23:29:41 2022] GET /identity/v3/projects/fd475051e9cd456abbb6cff649b374cc => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:29:43.210462 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86db04-6073-461c-a754-1e8598d8f1fe None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:43.211022 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86db04-6073-461c-a754-1e8598d8f1fe None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:43.211022 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86db04-6073-461c-a754-1e8598d8f1fe None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:43.215127 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 3/6] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:29:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 23:29:43.221080 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f67ba6-4534-42b0-a46a-a39bc9f404da None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:43.221457 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f67ba6-4534-42b0-a46a-a39bc9f404da None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:43.221632 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f67ba6-4534-42b0-a46a-a39bc9f404da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:43.240940 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-a5f67ba6-4534-42b0-a46a-a39bc9f404da None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:29:43.242431 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-a5f67ba6-4534-42b0-a46a-a39bc9f404da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:29:43.250220 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 4/7] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:29:43 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 17 23:29:43.512493 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa86db04-6073-461c-a754-1e8598d8f1fe None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:29:43.525136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa86db04-6073-461c-a754-1e8598d8f1fe None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:29:43.529588 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b3339-895c-4b4e-b211-bec0988bf467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:29:43.530178 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b3339-895c-4b4e-b211-bec0988bf467 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:43.530455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b3339-895c-4b4e-b211-bec0988bf467 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:43.530728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b3339-895c-4b4e-b211-bec0988bf467 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:43.538285 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015b3339-895c-4b4e-b211-bec0988bf467 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:29:43.540100 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015b3339-895c-4b4e-b211-bec0988bf467 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:29:43.548702 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 4/8] 10.222.0.26 () {58 vars in 1324 bytes} [Wed Aug 17 23:29:43 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 23:29:43.553968 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f67ba6-4534-42b0-a46a-a39bc9f404da None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:29:43.563452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f67ba6-4534-42b0-a46a-a39bc9f404da None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:29:43.567784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4186d6-73c8-42a9-a7df-ff31ed958b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:29:43.568434 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4186d6-73c8-42a9-a7df-ff31ed958b11 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:43.568713 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4186d6-73c8-42a9-a7df-ff31ed958b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:43.568983 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4186d6-73c8-42a9-a7df-ff31ed958b11 admin admin] PATH_INFO: `/v3/users/8f1a8fbc56c44c829c8dd8013a782220` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:43.570277 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4186d6-73c8-42a9-a7df-ff31ed958b11 admin admin] RBAC: Authorizing `identity:get_user(user_id=8f1a8fbc56c44c829c8dd8013a782220)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:29:43.572120 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4186d6-73c8-42a9-a7df-ff31ed958b11 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:29:43.573614 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 5/9] 10.222.0.26 () {56 vars in 1198 bytes} [Wed Aug 17 23:29:43 2022] GET /identity/v3/users/8f1a8fbc56c44c829c8dd8013a782220 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:29:45.512671 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb56a10-1982-4d49-b772-d715c523edf9 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:45.513187 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb56a10-1982-4d49-b772-d715c523edf9 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:45.513187 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb56a10-1982-4d49-b772-d715c523edf9 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:45.514583 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 5/10] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:29:45.519571 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e06d88-4ceb-43d3-8622-870c3ee50a11 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:45.519953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e06d88-4ceb-43d3-8622-870c3ee50a11 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:45.520204 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e06d88-4ceb-43d3-8622-870c3ee50a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:45.538053 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-a7e06d88-4ceb-43d3-8622-870c3ee50a11 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:29:45.539229 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-a7e06d88-4ceb-43d3-8622-870c3ee50a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:29:45.551073 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdaa4dc-e192-4c76-92a7-21a693c0b86e None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:45.551352 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdaa4dc-e192-4c76-92a7-21a693c0b86e None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:45.551621 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdaa4dc-e192-4c76-92a7-21a693c0b86e None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:45.552952 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 6/11] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 23:29:45.557936 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d7a23-98c5-4d63-8e90-891c501cdf54 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:45.558214 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d7a23-98c5-4d63-8e90-891c501cdf54 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:45.558481 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d7a23-98c5-4d63-8e90-891c501cdf54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:45.600790 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG passlib.handlers.bcrypt [None req-fb4d7a23-98c5-4d63-8e90-891c501cdf54 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=85834) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 17 23:29:45.601209 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG passlib.handlers.bcrypt [None req-fb4d7a23-98c5-4d63-8e90-891c501cdf54 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=85834) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 17 23:29:45.619941 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 6/12] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:29:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:29:45.647210 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-fb4d7a23-98c5-4d63-8e90-891c501cdf54 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:29:45.648792 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-fb4d7a23-98c5-4d63-8e90-891c501cdf54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:29:45.745127 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 7/13] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:29:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 23:29:45.887713 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e06d88-4ceb-43d3-8622-870c3ee50a11 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:29:45.911710 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:29:45.992930 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e06d88-4ceb-43d3-8622-870c3ee50a11 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:29:46.014383 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4d7a23-98c5-4d63-8e90-891c501cdf54 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:29:46.025916 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b20e9-bc22-4adb-afc2-3a9bea767df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:29:46.026549 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b20e9-bc22-4adb-afc2-3a9bea767df6 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:46.026830 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b20e9-bc22-4adb-afc2-3a9bea767df6 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:46.027108 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b20e9-bc22-4adb-afc2-3a9bea767df6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:46.039505 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b20e9-bc22-4adb-afc2-3a9bea767df6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:29:46.048548 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-d58b20e9-bc22-4adb-afc2-3a9bea767df6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:29:46.048548 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:29:46.048548 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:29:46.048548 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:29:46.048548 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:29:46.048548 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:29:46.048548 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:29:46.048548 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:29:46.048548 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:29:46.048548 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:29:46.048548 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:29:46.048548 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:29:46.048548 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:29:46.048548 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:29:46.048548 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:29:46.048548 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:29:46.049524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:29:46.049524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:29:46.049524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:29:46.049524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:29:46.049524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:29:46.049524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:29:46.049524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:29:46.049524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:29:46.049524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:29:46.049524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:29:46.049524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:29:46.049524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:29:46.049524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:29:46.049524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:29:46.049524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:29:46.049524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:29:46.050731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:29:46.050731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:29:46.050731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:29:46.050731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:29:46.050731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:29:46.050731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:29:46.050731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:29:46.050731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:29:46.050731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:29:46.050731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:29:46.050731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:29:46.050731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:29:46.050731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 7/14] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 17 23:29:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:29:46.055294 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b20e9-bc22-4adb-afc2-3a9bea767df6 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:29:46.115984 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4d7a23-98c5-4d63-8e90-891c501cdf54 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:29:46.118280 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:29:46.136673 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:29:46.153974 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f99fb-3bd3-4590-a83f-7a1d69ff3d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:29:46.155439 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f99fb-3bd3-4590-a83f-7a1d69ff3d2c None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:46.155783 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f99fb-3bd3-4590-a83f-7a1d69ff3d2c None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:46.156086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f99fb-3bd3-4590-a83f-7a1d69ff3d2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:46.156399 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b20e9-bc22-4adb-afc2-3a9bea767df6 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:29:46.188990 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eaafeb-11a9-447b-a522-5f4e6fa8c700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:29:46.189588 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eaafeb-11a9-447b-a522-5f4e6fa8c700 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:46.189865 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eaafeb-11a9-447b-a522-5f4e6fa8c700 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:46.190130 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eaafeb-11a9-447b-a522-5f4e6fa8c700 None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:46.190792 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eaafeb-11a9-447b-a522-5f4e6fa8c700 None admin] RBAC: Adding query filter params (name=admin) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:29:46.191090 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eaafeb-11a9-447b-a522-5f4e6fa8c700 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:29:46.192513 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eaafeb-11a9-447b-a522-5f4e6fa8c700 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:29:46.204411 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 8/15] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:29:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:29:46.209851 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eaafeb-11a9-447b-a522-5f4e6fa8c700 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:29:46.221260 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95f99fb-3bd3-4590-a83f-7a1d69ff3d2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:29:46.222795 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95f99fb-3bd3-4590-a83f-7a1d69ff3d2c None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:29:46.306742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eaafeb-11a9-447b-a522-5f4e6fa8c700 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:29:46.318961 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 8/16] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:29:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 17 23:29:46.324521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f99fb-3bd3-4590-a83f-7a1d69ff3d2c None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:29:46.328913 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:29:46.343081 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fb3af0-12ca-4cf3-bc76-1ed9bb311a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:29:46.343710 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb3af0-12ca-4cf3-bc76-1ed9bb311a5d None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:46.344079 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb3af0-12ca-4cf3-bc76-1ed9bb311a5d None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:46.344358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb3af0-12ca-4cf3-bc76-1ed9bb311a5d None admin] PATH_INFO: `/v3/users/8f1a8fbc56c44c829c8dd8013a782220` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:46.345506 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fb3af0-12ca-4cf3-bc76-1ed9bb311a5d None admin] RBAC: Authorizing `identity:get_user(user_id=8f1a8fbc56c44c829c8dd8013a782220)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:29:46.346971 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fb3af0-12ca-4cf3-bc76-1ed9bb311a5d None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:29:46.348693 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 9/17] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:29:46 2022] GET /identity/v3/users/8f1a8fbc56c44c829c8dd8013a782220 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:29:46.354082 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fb3af0-12ca-4cf3-bc76-1ed9bb311a5d None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:29:46.368548 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:29:46.424497 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f99fb-3bd3-4590-a83f-7a1d69ff3d2c None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:29:46.453257 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fb3af0-12ca-4cf3-bc76-1ed9bb311a5d None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:29:46.462125 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f6c3c-698f-4b05-90ec-c0cfe4995b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:29:46.462984 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f6c3c-698f-4b05-90ec-c0cfe4995b42 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:46.463265 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f6c3c-698f-4b05-90ec-c0cfe4995b42 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:46.463541 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f6c3c-698f-4b05-90ec-c0cfe4995b42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:46.482538 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f6c3c-698f-4b05-90ec-c0cfe4995b42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:29:46.484145 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f6c3c-698f-4b05-90ec-c0cfe4995b42 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:29:46.485961 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e7f069-a189-4f65-aec8-be5545e1790e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:29:46.486587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7f069-a189-4f65-aec8-be5545e1790e None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:46.486867 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7f069-a189-4f65-aec8-be5545e1790e None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:46.487141 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7f069-a189-4f65-aec8-be5545e1790e None admin] PATH_INFO: `/v3/domains/default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:46.488278 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e7f069-a189-4f65-aec8-be5545e1790e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:29:46.489723 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e7f069-a189-4f65-aec8-be5545e1790e None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:29:46.491301 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 10/18] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:29:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:29:46.497135 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e7f069-a189-4f65-aec8-be5545e1790e None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:29:46.502716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-d47f6c3c-698f-4b05-90ec-c0cfe4995b42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:29:46.502716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:29:46.502716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:29:46.502716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:29:46.502716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:29:46.502716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:29:46.502716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:29:46.502716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:29:46.502716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:29:46.502716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:29:46.502716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:29:46.502716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:29:46.502716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:29:46.502716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:29:46.502716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:29:46.502716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:29:46.503728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:29:46.503728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:29:46.503728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:29:46.503728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:29:46.503728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:29:46.503728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:29:46.503728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:29:46.503728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:29:46.503728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:29:46.503728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:29:46.503728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:29:46.503728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:29:46.503728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:29:46.503728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:29:46.503728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:29:46.503728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:29:46.503728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:29:46.504751 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:29:46.504751 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:29:46.504751 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:29:46.504751 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 9/19] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:29:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:29:46.509405 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f6c3c-698f-4b05-90ec-c0cfe4995b42 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:29:46.565799 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:29:46.578688 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:29:46.599575 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e7f069-a189-4f65-aec8-be5545e1790e None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:29:46.610829 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f6c3c-698f-4b05-90ec-c0cfe4995b42 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:29:46.632991 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ddfd3c-a902-4b17-92e0-567488468b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:29:46.633599 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ddfd3c-a902-4b17-92e0-567488468b98 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:46.633877 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ddfd3c-a902-4b17-92e0-567488468b98 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:46.634146 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ddfd3c-a902-4b17-92e0-567488468b98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:46.634873 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ddfd3c-a902-4b17-92e0-567488468b98 None admin] RBAC: Adding query filter params (role.id=33bd60d3b0e94dcbbfe97e87f91a8d78, scope.domain.id=default, user.id=8f1a8fbc56c44c829c8dd8013a782220) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:29:46.635180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ddfd3c-a902-4b17-92e0-567488468b98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:29:46.636666 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ddfd3c-a902-4b17-92e0-567488468b98 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:29:46.652043 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 11/20] 10.222.0.26 () {56 vars in 1322 bytes} [Wed Aug 17 23:29:46 2022] GET /identity/v3/role_assignments?user.id=8f1a8fbc56c44c829c8dd8013a782220&scope.domain.id=default&role.id=33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 233 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:29:46.652242 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02afa40-6658-4694-97eb-a66e65d52fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:29:46.652242 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02afa40-6658-4694-97eb-a66e65d52fb6 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:46.652242 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02afa40-6658-4694-97eb-a66e65d52fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:46.652242 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02afa40-6658-4694-97eb-a66e65d52fb6 None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:46.652541 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02afa40-6658-4694-97eb-a66e65d52fb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:29:46.652541 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02afa40-6658-4694-97eb-a66e65d52fb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:29:46.652541 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02afa40-6658-4694-97eb-a66e65d52fb6 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:29:46.669704 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 10/21] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:29:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:29:48.677630 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943680bc-8714-4a42-9988-8983ad0c921a None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:48.678741 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943680bc-8714-4a42-9988-8983ad0c921a None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:48.679221 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943680bc-8714-4a42-9988-8983ad0c921a None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:48.681198 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 12/22] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:29:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:29:48.687136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104c344-9b00-4c39-8a0a-0e89fc8c463b None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:48.687985 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104c344-9b00-4c39-8a0a-0e89fc8c463b None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:48.688652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104c344-9b00-4c39-8a0a-0e89fc8c463b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:48.712763 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-f104c344-9b00-4c39-8a0a-0e89fc8c463b None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:29:48.714266 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-f104c344-9b00-4c39-8a0a-0e89fc8c463b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:29:48.802028 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 11/23] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:29:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:29:49.035669 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10419eb6-cfa4-4c92-9803-7dc10cfb9536 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:49.036400 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10419eb6-cfa4-4c92-9803-7dc10cfb9536 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:49.036898 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10419eb6-cfa4-4c92-9803-7dc10cfb9536 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:49.038449 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 13/24] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:29:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:29:49.043716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227aa63-cb79-4a37-82ba-13a1fd3bbb4f None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:49.044030 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227aa63-cb79-4a37-82ba-13a1fd3bbb4f None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:49.044308 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227aa63-cb79-4a37-82ba-13a1fd3bbb4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:49.068925 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-4227aa63-cb79-4a37-82ba-13a1fd3bbb4f None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:29:49.070192 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-4227aa63-cb79-4a37-82ba-13a1fd3bbb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:29:49.126037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:29:49.159559 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 12/25] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:29:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:29:49.298006 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c53dd6-0384-434e-a545-a59a4ac55214 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:49.298191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4227aa63-cb79-4a37-82ba-13a1fd3bbb4f None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:29:49.298619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c53dd6-0384-434e-a545-a59a4ac55214 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:49.298904 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c53dd6-0384-434e-a545-a59a4ac55214 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:49.300663 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 14/26] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:29:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:29:49.306077 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8981e9-189e-4bc6-9287-43ab25af4c3b None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:49.306365 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8981e9-189e-4bc6-9287-43ab25af4c3b None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:49.306644 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8981e9-189e-4bc6-9287-43ab25af4c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:49.336935 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-1b8981e9-189e-4bc6-9287-43ab25af4c3b None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:29:49.338156 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-1b8981e9-189e-4bc6-9287-43ab25af4c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:29:49.417904 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4227aa63-cb79-4a37-82ba-13a1fd3bbb4f None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:29:49.454766 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce44111c-ddd0-4dc3-859f-4b3bfdae776f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:29:49.454766 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44111c-ddd0-4dc3-859f-4b3bfdae776f None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:49.454766 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44111c-ddd0-4dc3-859f-4b3bfdae776f None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:49.454766 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44111c-ddd0-4dc3-859f-4b3bfdae776f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:49.465563 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce44111c-ddd0-4dc3-859f-4b3bfdae776f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:29:49.472598 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-ce44111c-ddd0-4dc3-859f-4b3bfdae776f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:29:49.472598 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:29:49.472598 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:29:49.472598 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:29:49.472598 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:29:49.472598 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:29:49.472598 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:29:49.472598 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:29:49.472598 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:29:49.472598 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:29:49.472598 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:29:49.472598 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:29:49.472598 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:29:49.472598 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:29:49.472598 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:29:49.472598 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:29:49.473455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:29:49.473455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:29:49.473455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:29:49.473455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:29:49.473455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:29:49.473455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:29:49.473455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:29:49.473455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:29:49.473455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:29:49.473455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:29:49.473455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:29:49.473455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:29:49.473455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:29:49.473455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:29:49.473455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:29:49.473455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:29:49.474191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:29:49.474191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:29:49.474191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:29:49.474191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:29:49.474191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:29:49.474191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:29:49.474191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:29:49.474191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:29:49.474191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:29:49.474191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:29:49.474191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:29:49.474191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:29:49.474191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 13/27] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 17 23:29:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:29:49.479645 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce44111c-ddd0-4dc3-859f-4b3bfdae776f None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:29:49.506463 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 15/28] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:29:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:29:49.548474 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:29:49.583538 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce44111c-ddd0-4dc3-859f-4b3bfdae776f None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:29:49.610223 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873d07b-8e28-4a76-b246-066714af8954 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:49.610509 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873d07b-8e28-4a76-b246-066714af8954 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:49.610786 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873d07b-8e28-4a76-b246-066714af8954 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:49.612431 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 16/29] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:29:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 23:29:49.619661 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b65cd6d-1260-44cc-ad59-8807a98432c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:29:49.620437 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65cd6d-1260-44cc-ad59-8807a98432c8 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:49.620714 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65cd6d-1260-44cc-ad59-8807a98432c8 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:49.620988 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65cd6d-1260-44cc-ad59-8807a98432c8 None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:49.621415 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe4ee1-4045-41a1-9995-5779e2707d8b None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:49.621702 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe4ee1-4045-41a1-9995-5779e2707d8b None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:49.621797 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b65cd6d-1260-44cc-ad59-8807a98432c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:29:49.621976 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe4ee1-4045-41a1-9995-5779e2707d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:49.622078 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b65cd6d-1260-44cc-ad59-8807a98432c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:29:49.623418 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b65cd6d-1260-44cc-ad59-8807a98432c8 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:29:49.635784 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 14/30] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:29:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:29:49.641478 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b65cd6d-1260-44cc-ad59-8807a98432c8 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:29:49.661208 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-bcfe4ee1-4045-41a1-9995-5779e2707d8b None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:29:49.664434 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-bcfe4ee1-4045-41a1-9995-5779e2707d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:29:49.772317 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b65cd6d-1260-44cc-ad59-8807a98432c8 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:29:49.808721 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:29:49.813300 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 17/31] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:29:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 23:29:49.814149 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe4ee1-4045-41a1-9995-5779e2707d8b None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:29:49.819390 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:29:49.825180 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a68d1e-8244-4d0e-93a6-4b39c937dcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:29:49.825834 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a68d1e-8244-4d0e-93a6-4b39c937dcdf None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:49.826120 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a68d1e-8244-4d0e-93a6-4b39c937dcdf None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:49.830510 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a68d1e-8244-4d0e-93a6-4b39c937dcdf None admin] PATH_INFO: `/v3/users/8f1a8fbc56c44c829c8dd8013a782220` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:49.832863 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a68d1e-8244-4d0e-93a6-4b39c937dcdf None admin] RBAC: Authorizing `identity:get_user(user_id=8f1a8fbc56c44c829c8dd8013a782220)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:29:49.834590 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a68d1e-8244-4d0e-93a6-4b39c937dcdf None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:29:49.837821 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 15/32] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:29:49 2022] GET /identity/v3/users/8f1a8fbc56c44c829c8dd8013a782220 => generated 247 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:29:49.845202 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a68d1e-8244-4d0e-93a6-4b39c937dcdf None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:29:50.016108 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe4ee1-4045-41a1-9995-5779e2707d8b None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:29:50.042632 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a68d1e-8244-4d0e-93a6-4b39c937dcdf None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:29:50.082233 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a2c42-f401-40f9-86e7-f58404c3de78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:29:50.082782 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a2c42-f401-40f9-86e7-f58404c3de78 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:50.083112 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a2c42-f401-40f9-86e7-f58404c3de78 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:50.083388 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a2c42-f401-40f9-86e7-f58404c3de78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:50.103391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a2c42-f401-40f9-86e7-f58404c3de78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:29:50.106391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a2c42-f401-40f9-86e7-f58404c3de78 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:29:50.108625 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d46cf-5e8a-4e8e-b97c-5fe1f0af5d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:29:50.109274 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d46cf-5e8a-4e8e-b97c-5fe1f0af5d90 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:50.109549 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d46cf-5e8a-4e8e-b97c-5fe1f0af5d90 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:50.109817 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d46cf-5e8a-4e8e-b97c-5fe1f0af5d90 None admin] PATH_INFO: `/v3/domains/default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:50.111842 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96d46cf-5e8a-4e8e-b97c-5fe1f0af5d90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:29:50.114261 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96d46cf-5e8a-4e8e-b97c-5fe1f0af5d90 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:29:50.116098 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 16/33] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:29:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:29:50.116896 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d46cf-5e8a-4e8e-b97c-5fe1f0af5d90 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:29:50.131486 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-a57a2c42-f401-40f9-86e7-f58404c3de78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:29:50.131486 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:29:50.131486 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:29:50.131486 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:29:50.131486 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:29:50.131486 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:29:50.131486 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:29:50.131486 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:29:50.131486 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:29:50.131486 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:29:50.131486 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:29:50.131486 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:29:50.131486 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:29:50.131486 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:29:50.131486 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:29:50.131486 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:29:50.132399 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:29:50.132399 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:29:50.132399 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:29:50.132399 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:29:50.132399 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:29:50.132399 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:29:50.132399 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:29:50.132399 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:29:50.132399 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:29:50.132399 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:29:50.132399 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:29:50.132399 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:29:50.132399 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:29:50.132399 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:29:50.132399 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:29:50.132399 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:29:50.132399 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:29:50.133213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:29:50.133213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:29:50.133213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:29:50.133213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 18/34] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:29:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:29:50.133916 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a2c42-f401-40f9-86e7-f58404c3de78 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:29:50.153406 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:29:50.204390 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:29:50.221730 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d46cf-5e8a-4e8e-b97c-5fe1f0af5d90 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:29:50.247179 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a2c42-f401-40f9-86e7-f58404c3de78 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:29:50.256722 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e7cd5f-ef7a-4a8a-b1ee-4c0e44dd552d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:29:50.257347 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7cd5f-ef7a-4a8a-b1ee-4c0e44dd552d None admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:50.257631 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7cd5f-ef7a-4a8a-b1ee-4c0e44dd552d None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:50.257906 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7cd5f-ef7a-4a8a-b1ee-4c0e44dd552d None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:50.258579 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e7cd5f-ef7a-4a8a-b1ee-4c0e44dd552d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:29:50.259971 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e7cd5f-ef7a-4a8a-b1ee-4c0e44dd552d None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:29:50.288218 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ebf73-4678-4a3f-83e9-97332899272b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:29:50.289188 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ebf73-4678-4a3f-83e9-97332899272b None admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:50.289685 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ebf73-4678-4a3f-83e9-97332899272b None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:50.290146 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ebf73-4678-4a3f-83e9-97332899272b None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:50.291012 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ebf73-4678-4a3f-83e9-97332899272b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:29:50.292688 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ebf73-4678-4a3f-83e9-97332899272b None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:29:50.392725 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 17/35] 10.222.0.26 () {60 vars in 1131 bytes} [Wed Aug 17 23:29:50 2022] POST /identity/v3/roles => generated 219 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:29:50.393688 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e7cd5f-ef7a-4a8a-b1ee-4c0e44dd552d None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:29:50.420309 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 19/36] 10.222.0.26 () {60 vars in 1131 bytes} [Wed Aug 17 23:29:50 2022] POST /identity/v3/roles => generated 225 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:29:50.421874 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ebf73-4678-4a3f-83e9-97332899272b None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:29:50.508300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:29:50.514957 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e7cd5f-ef7a-4a8a-b1ee-4c0e44dd552d None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:29:50.557277 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ebf73-4678-4a3f-83e9-97332899272b None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:29:50.568872 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a983fefd-bbb8-423a-a02f-4f450ca114b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:29:50.569693 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a983fefd-bbb8-423a-a02f-4f450ca114b2 None admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:50.570048 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a983fefd-bbb8-423a-a02f-4f450ca114b2 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:50.570399 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a983fefd-bbb8-423a-a02f-4f450ca114b2 None admin] PATH_INFO: `/v3/domains/default/users/8f1a8fbc56c44c829c8dd8013a782220/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:50.577074 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a983fefd-bbb8-423a-a02f-4f450ca114b2 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=8f1a8fbc56c44c829c8dd8013a782220, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:29:50.578025 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a983fefd-bbb8-423a-a02f-4f450ca114b2 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:29:50.580992 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:29:50.602904 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 18/37] 10.222.0.26 () {58 vars in 1361 bytes} [Wed Aug 17 23:29:50 2022] PUT /identity/v3/domains/default/users/8f1a8fbc56c44c829c8dd8013a782220/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:29:50.607991 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f608af-e9d3-41d0-8ec8-d1032a4858d5 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:50.607991 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f608af-e9d3-41d0-8ec8-d1032a4858d5 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:50.607991 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f608af-e9d3-41d0-8ec8-d1032a4858d5 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:50.607991 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 19/38] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:29:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:29:50.616037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fdccf0-cc5c-47b7-95d5-0e3466c1a98e None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:50.616037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fdccf0-cc5c-47b7-95d5-0e3466c1a98e None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:50.616037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fdccf0-cc5c-47b7-95d5-0e3466c1a98e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:50.618378 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb7c001-3c0e-4f66-ab01-58dc2b73011d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:29:50.619208 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7c001-3c0e-4f66-ab01-58dc2b73011d None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:50.619685 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7c001-3c0e-4f66-ab01-58dc2b73011d None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:50.620167 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7c001-3c0e-4f66-ab01-58dc2b73011d None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:50.621168 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb7c001-3c0e-4f66-ab01-58dc2b73011d None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:29:50.622018 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb7c001-3c0e-4f66-ab01-58dc2b73011d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:29:50.625755 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb7c001-3c0e-4f66-ab01-58dc2b73011d None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:29:50.638529 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-76fdccf0-cc5c-47b7-95d5-0e3466c1a98e None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:29:50.641319 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-76fdccf0-cc5c-47b7-95d5-0e3466c1a98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:29:50.651902 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 20/39] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:29:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:29:50.657900 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb7c001-3c0e-4f66-ab01-58dc2b73011d None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:29:50.733029 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 20/40] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:29:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 23:29:50.735081 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e9571-3400-4f9d-9a02-209652a48963 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:50.735081 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e9571-3400-4f9d-9a02-209652a48963 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:50.735243 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e9571-3400-4f9d-9a02-209652a48963 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:50.736761 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 21/41] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:29:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:29:50.743045 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e433e-7233-49c0-aacc-38f512d17b72 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:50.743330 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e433e-7233-49c0-aacc-38f512d17b72 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:50.743611 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e433e-7233-49c0-aacc-38f512d17b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:50.770114 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-255e433e-7233-49c0-aacc-38f512d17b72 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:29:50.771331 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-255e433e-7233-49c0-aacc-38f512d17b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:29:50.807745 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb7c001-3c0e-4f66-ab01-58dc2b73011d None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:29:50.863901 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 22/42] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:29:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 23:29:50.954485 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3c49a-37f2-4907-b5a3-44bd2878d1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:29:50.955109 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3c49a-37f2-4907-b5a3-44bd2878d1f2 None admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:50.955282 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3c49a-37f2-4907-b5a3-44bd2878d1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:50.955557 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3c49a-37f2-4907-b5a3-44bd2878d1f2 None admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:50.956266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a3c49a-37f2-4907-b5a3-44bd2878d1f2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:29:50.957722 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a3c49a-37f2-4907-b5a3-44bd2878d1f2 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:29:50.994229 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 21/43] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Aug 17 23:29:50 2022] POST /identity/v3/projects => generated 301 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:29:51.031620 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e433e-7233-49c0-aacc-38f512d17b72 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:29:51.049072 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:29:51.147335 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e433e-7233-49c0-aacc-38f512d17b72 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:29:51.157862 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3c49a-37f2-4907-b5a3-44bd2878d1f2 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:29:51.184046 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56442f9-d25a-4085-898d-3b321551bab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:29:51.184668 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56442f9-d25a-4085-898d-3b321551bab4 None admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:51.184943 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56442f9-d25a-4085-898d-3b321551bab4 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:51.185220 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56442f9-d25a-4085-898d-3b321551bab4 None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:51.186490 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56442f9-d25a-4085-898d-3b321551bab4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:29:51.187888 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56442f9-d25a-4085-898d-3b321551bab4 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:29:51.192335 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:29:51.207635 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 23/44] 10.222.0.26 () {60 vars in 1131 bytes} [Wed Aug 17 23:29:51 2022] POST /identity/v3/roles => generated 223 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:29:51.284476 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3c49a-37f2-4907-b5a3-44bd2878d1f2 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:29:51.325641 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac1ac13-42fd-4e96-b202-765aecec8377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:29:51.326312 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1ac13-42fd-4e96-b202-765aecec8377 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:51.326594 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1ac13-42fd-4e96-b202-765aecec8377 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:51.326865 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1ac13-42fd-4e96-b202-765aecec8377 None admin] PATH_INFO: `/v3/domains/default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:51.328069 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac1ac13-42fd-4e96-b202-765aecec8377 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:29:51.329680 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac1ac13-42fd-4e96-b202-765aecec8377 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:29:51.331487 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 22/45] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:29:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:29:51.337065 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56442f9-d25a-4085-898d-3b321551bab4 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:29:51.419420 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:29:51.445099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56442f9-d25a-4085-898d-3b321551bab4 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:29:51.478162 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3e77c-1ec0-42fe-9172-8a17fb85b20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:29:51.478766 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3e77c-1ec0-42fe-9172-8a17fb85b20f None admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:51.479043 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3e77c-1ec0-42fe-9172-8a17fb85b20f None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:51.479312 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3e77c-1ec0-42fe-9172-8a17fb85b20f None admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:51.479948 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb3e77c-1ec0-42fe-9172-8a17fb85b20f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:29:51.481459 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb3e77c-1ec0-42fe-9172-8a17fb85b20f None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:29:51.509677 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 24/46] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Aug 17 23:29:51 2022] POST /identity/v3/projects => generated 312 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:29:52.630280 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8bca3-0802-4a40-803d-73d4b76d4a8a None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:52.630799 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8bca3-0802-4a40-803d-73d4b76d4a8a None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:52.630799 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8bca3-0802-4a40-803d-73d4b76d4a8a None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:52.632311 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 23/47] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:29:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:29:52.637336 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204ba15-5f09-4da5-8db2-3db701b2d459 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:52.637727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204ba15-5f09-4da5-8db2-3db701b2d459 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:52.637906 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204ba15-5f09-4da5-8db2-3db701b2d459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:52.656652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-6204ba15-5f09-4da5-8db2-3db701b2d459 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:29:52.657849 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-6204ba15-5f09-4da5-8db2-3db701b2d459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:29:52.724698 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 25/48] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:29:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:29:52.992299 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d8bca3-0802-4a40-803d-73d4b76d4a8a None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:29:53.086166 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:29:53.093817 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d8bca3-0802-4a40-803d-73d4b76d4a8a None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:29:53.127546 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c259cd-c36a-4103-b6f2-1ffc3f8194e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:29:53.128205 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c259cd-c36a-4103-b6f2-1ffc3f8194e6 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:53.128489 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c259cd-c36a-4103-b6f2-1ffc3f8194e6 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:53.128763 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c259cd-c36a-4103-b6f2-1ffc3f8194e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:53.139175 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c259cd-c36a-4103-b6f2-1ffc3f8194e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:29:53.143969 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-c6c259cd-c36a-4103-b6f2-1ffc3f8194e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:29:53.143969 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:29:53.143969 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:29:53.143969 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:29:53.143969 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:29:53.143969 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:29:53.143969 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:29:53.143969 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:29:53.143969 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:29:53.143969 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:29:53.143969 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:29:53.143969 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:29:53.143969 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:29:53.143969 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:29:53.143969 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:29:53.143969 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:29:53.144951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:29:53.144951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:29:53.144951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:29:53.144951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:29:53.144951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:29:53.144951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:29:53.144951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:29:53.144951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:29:53.144951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:29:53.144951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:29:53.144951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:29:53.144951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:29:53.144951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:29:53.144951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:29:53.144951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:29:53.144951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:29:53.145926 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:29:53.145926 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:29:53.145926 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:29:53.145926 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:29:53.145926 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:29:53.145926 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:29:53.145926 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:29:53.145926 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:29:53.145926 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:29:53.145926 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:29:53.145926 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:29:53.145926 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:29:53.145926 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 24/49] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 17 23:29:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:29:53.152551 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204ba15-5f09-4da5-8db2-3db701b2d459 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:29:53.309916 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:29:53.356643 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204ba15-5f09-4da5-8db2-3db701b2d459 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:29:53.391402 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec0aed-136b-4193-8521-67b6cd65ce8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:29:53.392049 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec0aed-136b-4193-8521-67b6cd65ce8a None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:53.392338 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec0aed-136b-4193-8521-67b6cd65ce8a None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:53.392616 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec0aed-136b-4193-8521-67b6cd65ce8a None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:53.393297 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbec0aed-136b-4193-8521-67b6cd65ce8a None admin] RBAC: Adding query filter params (name=admin) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:29:53.393607 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbec0aed-136b-4193-8521-67b6cd65ce8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:29:53.395006 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbec0aed-136b-4193-8521-67b6cd65ce8a None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:29:53.398028 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050c26b-b51d-4de1-b16b-a82674e6e7b8 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:53.398315 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050c26b-b51d-4de1-b16b-a82674e6e7b8 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:53.398590 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050c26b-b51d-4de1-b16b-a82674e6e7b8 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:53.399950 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 25/50] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 23:29:53.403668 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 26/51] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:29:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:29:53.405336 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf3f5b-56b8-4d91-8dff-0dedc2cc0746 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:53.405624 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf3f5b-56b8-4d91-8dff-0dedc2cc0746 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:53.405901 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf3f5b-56b8-4d91-8dff-0dedc2cc0746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:53.408756 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec0aed-136b-4193-8521-67b6cd65ce8a None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:29:53.423413 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-0fdf3f5b-56b8-4d91-8dff-0dedc2cc0746 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:29:53.425000 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-0fdf3f5b-56b8-4d91-8dff-0dedc2cc0746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:29:53.495573 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 26/52] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:29:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:29:53.507910 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec0aed-136b-4193-8521-67b6cd65ce8a None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:29:53.524039 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:29:53.540997 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03626572-bc0b-4981-8662-7b4d6d12334e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:29:53.541629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03626572-bc0b-4981-8662-7b4d6d12334e None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:53.541915 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03626572-bc0b-4981-8662-7b4d6d12334e None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:53.542196 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03626572-bc0b-4981-8662-7b4d6d12334e None admin] PATH_INFO: `/v3/users/8f1a8fbc56c44c829c8dd8013a782220` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:53.543345 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03626572-bc0b-4981-8662-7b4d6d12334e None admin] RBAC: Authorizing `identity:get_user(user_id=8f1a8fbc56c44c829c8dd8013a782220)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:29:53.545350 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03626572-bc0b-4981-8662-7b4d6d12334e None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:29:53.547274 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 27/53] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:29:53 2022] GET /identity/v3/users/8f1a8fbc56c44c829c8dd8013a782220 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:29:53.552888 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf3f5b-56b8-4d91-8dff-0dedc2cc0746 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:29:53.636238 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:29:53.650931 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf3f5b-56b8-4d91-8dff-0dedc2cc0746 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:29:53.684633 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f45f9b0-16eb-41a5-bd94-1b9411506986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:29:53.685261 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45f9b0-16eb-41a5-bd94-1b9411506986 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:53.686088 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45f9b0-16eb-41a5-bd94-1b9411506986 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:53.686366 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45f9b0-16eb-41a5-bd94-1b9411506986 None admin] PATH_INFO: `/v3/domains/default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:53.687492 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f45f9b0-16eb-41a5-bd94-1b9411506986 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:29:53.688978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f45f9b0-16eb-41a5-bd94-1b9411506986 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:29:53.690714 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 27/54] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:29:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:29:53.696285 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03626572-bc0b-4981-8662-7b4d6d12334e None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:29:53.765767 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f45f9b0-16eb-41a5-bd94-1b9411506986 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:29:53.795038 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03626572-bc0b-4981-8662-7b4d6d12334e None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:29:53.827889 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd69f3-b82f-4233-937a-289378f1e9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:29:53.828530 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd69f3-b82f-4233-937a-289378f1e9be None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:53.828814 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd69f3-b82f-4233-937a-289378f1e9be None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:53.829091 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd69f3-b82f-4233-937a-289378f1e9be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:53.829820 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cd69f3-b82f-4233-937a-289378f1e9be None admin] RBAC: Adding query filter params (role.id=33bd60d3b0e94dcbbfe97e87f91a8d78, scope.domain.id=default, user.id=8f1a8fbc56c44c829c8dd8013a782220) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:29:53.830129 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cd69f3-b82f-4233-937a-289378f1e9be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:29:53.831581 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cd69f3-b82f-4233-937a-289378f1e9be None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:29:53.839891 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 28/55] 10.222.0.26 () {56 vars in 1322 bytes} [Wed Aug 17 23:29:53 2022] GET /identity/v3/role_assignments?user.id=8f1a8fbc56c44c829c8dd8013a782220&scope.domain.id=default&role.id=33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 530 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:29:53.872337 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f45f9b0-16eb-41a5-bd94-1b9411506986 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:29:53.908885 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f01b9-e4d8-40cd-aa19-8d430675291a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:29:53.909443 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f01b9-e4d8-40cd-aa19-8d430675291a None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:53.909722 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f01b9-e4d8-40cd-aa19-8d430675291a None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:53.909995 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f01b9-e4d8-40cd-aa19-8d430675291a None admin] PATH_INFO: `/v3/domains/default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:53.911131 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80f01b9-e4d8-40cd-aa19-8d430675291a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:29:53.912653 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80f01b9-e4d8-40cd-aa19-8d430675291a None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:29:53.914290 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 28/56] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:29:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:29:53.920244 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd69f3-b82f-4233-937a-289378f1e9be None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:29:53.977282 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:29:54.029049 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd69f3-b82f-4233-937a-289378f1e9be None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:29:54.062470 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f939edd-d026-440a-9edb-bf8c93b8522b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:29:54.063073 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f939edd-d026-440a-9edb-bf8c93b8522b None admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:54.063353 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f939edd-d026-440a-9edb-bf8c93b8522b None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:54.063625 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f939edd-d026-440a-9edb-bf8c93b8522b None admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:54.064306 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f939edd-d026-440a-9edb-bf8c93b8522b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:29:54.065753 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f939edd-d026-440a-9edb-bf8c93b8522b None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:29:54.090794 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 29/57] 10.222.0.26 () {60 vars in 1140 bytes} [Wed Aug 17 23:29:53 2022] POST /identity/v3/projects => generated 298 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:29:55.955152 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe613d6-03bd-4459-975f-99d0f8b09a86 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:55.955686 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe613d6-03bd-4459-975f-99d0f8b09a86 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:55.955686 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe613d6-03bd-4459-975f-99d0f8b09a86 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:55.957083 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 29/58] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:29:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:29:55.962179 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a8526-2d3b-46e2-b643-70b341b2a9f7 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:55.962573 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a8526-2d3b-46e2-b643-70b341b2a9f7 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:55.962741 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a8526-2d3b-46e2-b643-70b341b2a9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:55.980820 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-b42a8526-2d3b-46e2-b643-70b341b2a9f7 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:29:55.982013 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-b42a8526-2d3b-46e2-b643-70b341b2a9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:29:56.046918 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 30/59] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:29:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:29:56.312323 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe613d6-03bd-4459-975f-99d0f8b09a86 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:29:56.337479 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:29:56.412165 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe613d6-03bd-4459-975f-99d0f8b09a86 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:29:56.445369 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb0332-8e6f-439f-b847-d69bb35686d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:29:56.445992 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb0332-8e6f-439f-b847-d69bb35686d1 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:56.446269 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb0332-8e6f-439f-b847-d69bb35686d1 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:56.446538 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb0332-8e6f-439f-b847-d69bb35686d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:56.447681 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cb0332-8e6f-439f-b847-d69bb35686d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:29:56.449159 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cb0332-8e6f-439f-b847-d69bb35686d1 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:29:56.450733 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 30/60] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:29:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:29:56.456695 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a8526-2d3b-46e2-b643-70b341b2a9f7 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:29:56.486527 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:29:56.553238 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a8526-2d3b-46e2-b643-70b341b2a9f7 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:29:56.586118 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb79c96-0a27-4902-9c43-5720e543ef59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:29:56.586663 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb79c96-0a27-4902-9c43-5720e543ef59 None admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:56.586941 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb79c96-0a27-4902-9c43-5720e543ef59 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:56.587209 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb79c96-0a27-4902-9c43-5720e543ef59 None admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:56.587851 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb79c96-0a27-4902-9c43-5720e543ef59 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:29:56.589316 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb79c96-0a27-4902-9c43-5720e543ef59 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:29:56.614527 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 31/61] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Aug 17 23:29:56 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:29:59.461431 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593a350-4975-4aa2-ba4d-97a33279f2f1 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:59.462306 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593a350-4975-4aa2-ba4d-97a33279f2f1 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:59.462779 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593a350-4975-4aa2-ba4d-97a33279f2f1 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:59.464681 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 31/62] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:29:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:29:59.470031 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c371aac-777b-49ed-b4fc-f18553e5c684 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:59.470413 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c371aac-777b-49ed-b4fc-f18553e5c684 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:59.470590 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c371aac-777b-49ed-b4fc-f18553e5c684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:59.500041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-1c371aac-777b-49ed-b4fc-f18553e5c684 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:29:59.501258 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-1c371aac-777b-49ed-b4fc-f18553e5c684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:29:59.594526 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 32/63] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:29:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:29:59.627998 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9716d-4075-43f1-a520-1b0601acbc6a None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:59.627998 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9716d-4075-43f1-a520-1b0601acbc6a None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:59.627998 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9716d-4075-43f1-a520-1b0601acbc6a None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:59.630017 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 32/64] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:29:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:29:59.635013 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab6849-e09c-406e-bfc3-937f4cfd4fe2 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:29:59.635304 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab6849-e09c-406e-bfc3-937f4cfd4fe2 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:29:59.635586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab6849-e09c-406e-bfc3-937f4cfd4fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:29:59.658556 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-65ab6849-e09c-406e-bfc3-937f4cfd4fe2 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:29:59.659754 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-65ab6849-e09c-406e-bfc3-937f4cfd4fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:29:59.680037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:29:59.748802 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 33/65] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:29:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:29:59.935872 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b9716d-4075-43f1-a520-1b0601acbc6a None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:00.050755 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:00.062396 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b9716d-4075-43f1-a520-1b0601acbc6a None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:00.067739 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab6849-e09c-406e-bfc3-937f4cfd4fe2 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:00.118307 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e2f4a4-eaac-407b-8f95-a8179b5b0578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:00.119405 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2f4a4-eaac-407b-8f95-a8179b5b0578 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:00.119689 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2f4a4-eaac-407b-8f95-a8179b5b0578 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:00.119988 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2f4a4-eaac-407b-8f95-a8179b5b0578 None admin] PATH_INFO: `/v3/roles/member` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:00.135174 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e2f4a4-eaac-407b-8f95-a8179b5b0578 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:00.142892 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-d0e2f4a4-eaac-407b-8f95-a8179b5b0578 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 23:30:00.142892 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:00.142892 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:00.142892 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:00.142892 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:00.142892 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:00.142892 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:00.142892 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:00.142892 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:00.142892 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:00.142892 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:00.142892 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:00.142892 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:00.142892 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:00.142892 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:00.142892 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:00.143738 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:00.143738 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:00.143738 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:00.143738 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:00.143738 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:00.143738 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:00.143738 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:00.143738 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:00.143738 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:00.143738 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:00.143738 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:00.143738 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:00.143738 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:00.143738 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:00.143738 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:00.143738 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:00.144550 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:00.144550 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:00.144550 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:00.144550 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:00.144550 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:00.144550 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:00.144550 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:00.144550 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:00.144550 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:00.144550 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:00.144550 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 23:30:00.144550 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:30:00.144550 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 33/66] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:29:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:00.145656 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c57f667-f31e-466e-8562-8ddc764b5335 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:00.145810 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c57f667-f31e-466e-8562-8ddc764b5335 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:00.147994 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c57f667-f31e-466e-8562-8ddc764b5335 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:00.148569 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 34/67] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:00.151800 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c57f667-f31e-466e-8562-8ddc764b5335 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:00.181729 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab6849-e09c-406e-bfc3-937f4cfd4fe2 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:00.216333 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d69e4e-75c2-4e9f-aaea-c606cabe1a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:00.216999 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d69e4e-75c2-4e9f-aaea-c606cabe1a2d None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:00.217282 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d69e4e-75c2-4e9f-aaea-c606cabe1a2d None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:00.217561 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d69e4e-75c2-4e9f-aaea-c606cabe1a2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:00.227916 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d69e4e-75c2-4e9f-aaea-c606cabe1a2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:00.233325 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-02d69e4e-75c2-4e9f-aaea-c606cabe1a2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:30:00.233325 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:00.233325 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:00.233325 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:00.233325 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:00.233325 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:00.233325 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:00.233325 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:00.233325 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:00.233325 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:00.233325 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:00.233325 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:00.233325 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:00.233325 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:00.233325 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:00.233325 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:00.234183 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:00.234183 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:00.234183 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:00.234183 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:00.234183 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:00.234183 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:00.234183 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:00.234183 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:00.234183 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:00.234183 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:00.234183 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:00.234183 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:00.234183 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:00.234183 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:00.234183 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:00.234183 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:00.234956 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:00.234956 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:00.234956 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:00.234956 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:00.234956 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:00.234956 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:00.234956 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:00.234956 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:00.234956 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:00.234956 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:00.234956 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:30:00.234956 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:30:00.234956 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 34/68] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 17 23:30:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:30:00.236114 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b30e0c-8272-4238-813a-f9109da42bb4 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:00.237029 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b30e0c-8272-4238-813a-f9109da42bb4 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:00.237029 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b30e0c-8272-4238-813a-f9109da42bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:00.254623 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-65b30e0c-8272-4238-813a-f9109da42bb4 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:00.255825 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-65b30e0c-8272-4238-813a-f9109da42bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:00.257892 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:00.273274 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c57f667-f31e-466e-8562-8ddc764b5335 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:00.310022 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba41fd-5be4-4c46-969f-63fc1a3d40bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:00.311106 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba41fd-5be4-4c46-969f-63fc1a3d40bd None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:00.311386 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba41fd-5be4-4c46-969f-63fc1a3d40bd None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:00.311658 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba41fd-5be4-4c46-969f-63fc1a3d40bd None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:00.312463 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba41fd-5be4-4c46-969f-63fc1a3d40bd None admin] RBAC: Adding query filter params (name=member) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:00.312767 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba41fd-5be4-4c46-969f-63fc1a3d40bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:00.314173 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba41fd-5be4-4c46-969f-63fc1a3d40bd None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:00.322369 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 35/69] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 17 23:30:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:00.323532 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d66c7f-9be2-41e8-a961-ecb8e0029169 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:00.323816 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d66c7f-9be2-41e8-a961-ecb8e0029169 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:00.324121 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d66c7f-9be2-41e8-a961-ecb8e0029169 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:00.329389 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 36/70] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:00.329389 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d66c7f-9be2-41e8-a961-ecb8e0029169 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:00.336981 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 35/71] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:00.345212 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b30e0c-8272-4238-813a-f9109da42bb4 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:00.352337 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:00.433659 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d66c7f-9be2-41e8-a961-ecb8e0029169 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:00.461358 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b30e0c-8272-4238-813a-f9109da42bb4 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:00.473291 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9051d6-ded0-4867-aff9-194a3c0f1c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:00.473901 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9051d6-ded0-4867-aff9-194a3c0f1c63 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:00.474182 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9051d6-ded0-4867-aff9-194a3c0f1c63 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:00.474460 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9051d6-ded0-4867-aff9-194a3c0f1c63 None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:00.475439 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9051d6-ded0-4867-aff9-194a3c0f1c63 None admin] RBAC: Adding query filter params (name=admin) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:00.475439 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9051d6-ded0-4867-aff9-194a3c0f1c63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:00.476815 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9051d6-ded0-4867-aff9-194a3c0f1c63 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:00.490016 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 37/72] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:30:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:00.490016 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00aff8-7034-4a79-a005-343a25ecb373 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:00.490016 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00aff8-7034-4a79-a005-343a25ecb373 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:00.490016 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00aff8-7034-4a79-a005-343a25ecb373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:00.501431 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73143de1-b974-4ab0-959f-5a7406b6ffc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:00.502074 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73143de1-b974-4ab0-959f-5a7406b6ffc4 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:00.502357 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73143de1-b974-4ab0-959f-5a7406b6ffc4 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:00.502632 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73143de1-b974-4ab0-959f-5a7406b6ffc4 None admin] PATH_INFO: `/v3/users/45f6f1fc05fd4bf590127b863244444f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:00.506091 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-1a00aff8-7034-4a79-a005-343a25ecb373 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:00.507291 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-1a00aff8-7034-4a79-a005-343a25ecb373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:00.510881 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:00.519252 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73143de1-b974-4ab0-959f-5a7406b6ffc4 None admin] RBAC: Authorizing `identity:get_user(user_id=45f6f1fc05fd4bf590127b863244444f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:00.520863 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73143de1-b974-4ab0-959f-5a7406b6ffc4 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:00.522610 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 36/73] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:30:00 2022] GET /identity/v3/users/45f6f1fc05fd4bf590127b863244444f => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:00.525518 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48dacc-3d64-47b6-ab97-f85d7dcfa136 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:00.526384 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48dacc-3d64-47b6-ab97-f85d7dcfa136 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:00.526826 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48dacc-3d64-47b6-ab97-f85d7dcfa136 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:00.528608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 37/74] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:00.530041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48dacc-3d64-47b6-ab97-f85d7dcfa136 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:00.578455 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 38/75] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:00.579059 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00aff8-7034-4a79-a005-343a25ecb373 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:00.582026 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:00.648721 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48dacc-3d64-47b6-ab97-f85d7dcfa136 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:00.682303 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00aff8-7034-4a79-a005-343a25ecb373 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:00.687359 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab35b47d-d06a-420b-8437-b8a44d033b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:00.688302 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35b47d-d06a-420b-8437-b8a44d033b2d None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:00.688777 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35b47d-d06a-420b-8437-b8a44d033b2d None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:00.689369 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35b47d-d06a-420b-8437-b8a44d033b2d None admin] PATH_INFO: `/v3/users/8f1a8fbc56c44c829c8dd8013a782220` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:00.690556 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab35b47d-d06a-420b-8437-b8a44d033b2d None admin] RBAC: Authorizing `identity:get_user(user_id=8f1a8fbc56c44c829c8dd8013a782220)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:00.692172 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab35b47d-d06a-420b-8437-b8a44d033b2d None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:00.694008 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 38/76] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:30:00 2022] GET /identity/v3/users/8f1a8fbc56c44c829c8dd8013a782220 => generated 247 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:00.695625 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828fcc0-22de-4c29-bbcc-708608812ddb None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:00.695625 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828fcc0-22de-4c29-bbcc-708608812ddb None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:00.695625 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828fcc0-22de-4c29-bbcc-708608812ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:00.716240 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-d828fcc0-22de-4c29-bbcc-708608812ddb None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:00.718155 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-d828fcc0-22de-4c29-bbcc-708608812ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:00.721894 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0956157-0475-4056-b2be-5cc7f7a5ab28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:00.722502 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0956157-0475-4056-b2be-5cc7f7a5ab28 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:00.722780 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0956157-0475-4056-b2be-5cc7f7a5ab28 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:00.723050 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0956157-0475-4056-b2be-5cc7f7a5ab28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:00.778591 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:00.788702 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 39/77] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:00.789365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d828fcc0-22de-4c29-bbcc-708608812ddb None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:00.800109 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0956157-0475-4056-b2be-5cc7f7a5ab28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:00.801869 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0956157-0475-4056-b2be-5cc7f7a5ab28 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:00.804871 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:00.894146 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d828fcc0-22de-4c29-bbcc-708608812ddb None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:00.909573 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 39/78] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:30:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:30:00.910421 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0956157-0475-4056-b2be-5cc7f7a5ab28 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:00.934231 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5556df5-62c9-41a9-ae99-79252f3de8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:00.934869 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5556df5-62c9-41a9-ae99-79252f3de8af None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:00.935172 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5556df5-62c9-41a9-ae99-79252f3de8af None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:00.935444 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5556df5-62c9-41a9-ae99-79252f3de8af None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:00.946602 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5556df5-62c9-41a9-ae99-79252f3de8af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:00.951136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-d5556df5-62c9-41a9-ae99-79252f3de8af None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 23:30:00.951136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:00.951136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:00.951136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:00.951136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:00.951136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:00.951136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:00.951136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:00.951136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:00.951136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:00.951136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:00.951136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:00.951136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:00.951136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:00.951136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:00.951136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:00.952010 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:00.952010 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:00.952010 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:00.952010 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:00.952010 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:00.952010 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:00.952010 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:00.952010 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:00.952010 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:00.952010 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:00.952010 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:00.952010 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:00.952010 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:00.952010 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:00.952010 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:00.952010 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:00.952801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:00.952801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:00.952801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:00.952801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:00.952801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:00.952801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:00.952801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:00.952801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:00.952801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:00.952801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:00.952801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 23:30:00.952801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:30:00.952801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 40/79] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 17 23:30:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:00.953452 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5556df5-62c9-41a9-ae99-79252f3de8af None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:01.008176 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:01.032518 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0956157-0475-4056-b2be-5cc7f7a5ab28 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:01.035704 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:01.057685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5556df5-62c9-41a9-ae99-79252f3de8af None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:01.068580 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae57fb-a84a-423f-9d1f-578820d2f20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:01.069192 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae57fb-a84a-423f-9d1f-578820d2f20e None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:01.069473 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae57fb-a84a-423f-9d1f-578820d2f20e None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:01.069748 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae57fb-a84a-423f-9d1f-578820d2f20e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:01.091174 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f36a87b-e08f-43b6-8982-35f961f6f9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:01.092371 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36a87b-e08f-43b6-8982-35f961f6f9a9 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:01.092659 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36a87b-e08f-43b6-8982-35f961f6f9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:01.092937 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36a87b-e08f-43b6-8982-35f961f6f9a9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:01.094060 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f36a87b-e08f-43b6-8982-35f961f6f9a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:01.095557 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f36a87b-e08f-43b6-8982-35f961f6f9a9 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:01.097170 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 41/80] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:30:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:01.100821 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f36a87b-e08f-43b6-8982-35f961f6f9a9 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:01.136452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ae57fb-a84a-423f-9d1f-578820d2f20e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:01.137939 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ae57fb-a84a-423f-9d1f-578820d2f20e None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:01.196582 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f36a87b-e08f-43b6-8982-35f961f6f9a9 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:01.219370 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:01.231668 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e7b24-da80-4497-a051-cee7468d9e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:01.232361 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e7b24-da80-4497-a051-cee7468d9e28 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:01.232646 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e7b24-da80-4497-a051-cee7468d9e28 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:01.232921 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e7b24-da80-4497-a051-cee7468d9e28 None admin] PATH_INFO: `/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:01.234049 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e7b24-da80-4497-a051-cee7468d9e28 None admin] RBAC: Authorizing `identity:get_project(project_id=105a2b33e5ee4cf88b11c48aebf329d7)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:01.239443 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 40/81] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:30:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:30:01.239573 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e7b24-da80-4497-a051-cee7468d9e28 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:01.239669 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae57fb-a84a-423f-9d1f-578820d2f20e None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:01.239751 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 42/82] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:01 2022] GET /identity/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7 => generated 298 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:01.239751 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e7b24-da80-4497-a051-cee7468d9e28 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:01.243104 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:01.334670 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e7b24-da80-4497-a051-cee7468d9e28 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:01.336480 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae57fb-a84a-423f-9d1f-578820d2f20e None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:01.368011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca1160-01e1-43b9-a1dc-88380c6e5b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:01.368636 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca1160-01e1-43b9-a1dc-88380c6e5b34 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:01.368917 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca1160-01e1-43b9-a1dc-88380c6e5b34 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:01.369193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca1160-01e1-43b9-a1dc-88380c6e5b34 None admin] PATH_INFO: `/v3/roles/member` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:01.370269 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dafe9b-c1f8-4c20-aec6-0107c90f6ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:01.370872 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dafe9b-c1f8-4c20-aec6-0107c90f6ffd None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:01.371154 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dafe9b-c1f8-4c20-aec6-0107c90f6ffd None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:01.371423 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dafe9b-c1f8-4c20-aec6-0107c90f6ffd None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:01.372130 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dafe9b-c1f8-4c20-aec6-0107c90f6ffd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:01.372438 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dafe9b-c1f8-4c20-aec6-0107c90f6ffd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:01.374351 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dafe9b-c1f8-4c20-aec6-0107c90f6ffd None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:01.379392 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ca1160-01e1-43b9-a1dc-88380c6e5b34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:01.382221 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 41/83] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:30:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:01.382808 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dafe9b-c1f8-4c20-aec6-0107c90f6ffd None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:01.384236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-06ca1160-01e1-43b9-a1dc-88380c6e5b34 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 23:30:01.384236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:01.384236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:01.384236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:01.384236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:01.384236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:01.384236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:01.384236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:01.384236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:01.384236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:01.384236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:01.384236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:01.384236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:01.384236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:01.384236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:01.384236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:01.385066 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:01.385066 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:01.385066 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:01.385066 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:01.385066 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:01.385066 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:01.385066 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:01.385066 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:01.385066 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:01.385066 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:01.385066 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:01.385066 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:01.385066 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:01.385066 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:01.385066 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:01.385066 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:01.385844 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:01.385844 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:01.385844 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:01.385844 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:01.385844 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:01.385844 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:01.385844 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:01.385844 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:01.385844 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:01.385844 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:01.385844 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 23:30:01.385844 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:30:01.385844 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 43/84] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:30:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:01.386560 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca1160-01e1-43b9-a1dc-88380c6e5b34 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:01.434520 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:01.455353 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:01.485089 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca1160-01e1-43b9-a1dc-88380c6e5b34 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:01.490547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dafe9b-c1f8-4c20-aec6-0107c90f6ffd None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:01.520251 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf56586-ddcd-4184-9724-81bacd06f61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:01.520880 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf56586-ddcd-4184-9724-81bacd06f61e None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:01.521163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf56586-ddcd-4184-9724-81bacd06f61e None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:01.521442 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf56586-ddcd-4184-9724-81bacd06f61e None admin] PATH_INFO: `/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:01.522570 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf56586-ddcd-4184-9724-81bacd06f61e None admin] RBAC: Authorizing `identity:get_project(project_id=105a2b33e5ee4cf88b11c48aebf329d7)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:01.523858 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72f26c5-7ba3-4a0b-b214-17c690a509e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:01.524048 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf56586-ddcd-4184-9724-81bacd06f61e None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:01.524488 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f26c5-7ba3-4a0b-b214-17c690a509e7 None admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:01.524767 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f26c5-7ba3-4a0b-b214-17c690a509e7 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:01.525042 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f26c5-7ba3-4a0b-b214-17c690a509e7 None admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:01.525685 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72f26c5-7ba3-4a0b-b214-17c690a509e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:01.525766 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 44/85] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:01 2022] GET /identity/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7 => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:01.526473 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf56586-ddcd-4184-9724-81bacd06f61e None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:01.527216 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72f26c5-7ba3-4a0b-b214-17c690a509e7 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:01.555313 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 42/86] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Aug 17 23:30:01 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:30:01.556132 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72f26c5-7ba3-4a0b-b214-17c690a509e7 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:01.635225 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf56586-ddcd-4184-9724-81bacd06f61e None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:01.655706 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:01.674552 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faece92-11b3-4ba9-9115-ecf5eedcebb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:01.675174 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faece92-11b3-4ba9-9115-ecf5eedcebb2 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:01.675467 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faece92-11b3-4ba9-9115-ecf5eedcebb2 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:01.675743 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faece92-11b3-4ba9-9115-ecf5eedcebb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:01.676711 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faece92-11b3-4ba9-9115-ecf5eedcebb2 None admin] RBAC: Adding query filter params (role.id=1182317a465349478dde4e28cb7cf96f, scope.project.id=105a2b33e5ee4cf88b11c48aebf329d7, user.id=45f6f1fc05fd4bf590127b863244444f) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:01.676867 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faece92-11b3-4ba9-9115-ecf5eedcebb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:01.678322 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faece92-11b3-4ba9-9115-ecf5eedcebb2 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:01.680144 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72f26c5-7ba3-4a0b-b214-17c690a509e7 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:01.686373 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 45/87] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:30:01 2022] GET /identity/v3/role_assignments?user.id=45f6f1fc05fd4bf590127b863244444f&scope.project.id=105a2b33e5ee4cf88b11c48aebf329d7&role.id=1182317a465349478dde4e28cb7cf96f => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:01.687025 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faece92-11b3-4ba9-9115-ecf5eedcebb2 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:01.722011 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:01.741371 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0b8181-5581-4225-b890-80fff7ae1ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:01.743117 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b8181-5581-4225-b890-80fff7ae1ef0 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:01.743621 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b8181-5581-4225-b890-80fff7ae1ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:01.744071 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b8181-5581-4225-b890-80fff7ae1ef0 None admin] PATH_INFO: `/v3/users/45f6f1fc05fd4bf590127b863244444f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:01.746370 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0b8181-5581-4225-b890-80fff7ae1ef0 None admin] RBAC: Authorizing `identity:get_user(user_id=45f6f1fc05fd4bf590127b863244444f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:01.748130 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0b8181-5581-4225-b890-80fff7ae1ef0 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:01.750060 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 43/88] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:30:01 2022] GET /identity/v3/users/45f6f1fc05fd4bf590127b863244444f => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:01.751032 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0b8181-5581-4225-b890-80fff7ae1ef0 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:01.831509 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faece92-11b3-4ba9-9115-ecf5eedcebb2 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:01.879190 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46650280-bc89-4ae3-9f66-72945b53beff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:01.880219 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46650280-bc89-4ae3-9f66-72945b53beff None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:01.880647 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46650280-bc89-4ae3-9f66-72945b53beff None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:01.881105 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46650280-bc89-4ae3-9f66-72945b53beff None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:01.881926 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46650280-bc89-4ae3-9f66-72945b53beff None admin] RBAC: Adding query filter params (name=member) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:01.882375 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46650280-bc89-4ae3-9f66-72945b53beff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:01.884174 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46650280-bc89-4ae3-9f66-72945b53beff None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:01.889366 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0b8181-5581-4225-b890-80fff7ae1ef0 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:01.893392 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 46/89] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 17 23:30:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:01.894228 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46650280-bc89-4ae3-9f66-72945b53beff None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:01.939113 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e12c2e8-e99b-4f5e-b3e0-6e1a01b55dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:01.939726 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e12c2e8-e99b-4f5e-b3e0-6e1a01b55dc9 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:01.940039 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e12c2e8-e99b-4f5e-b3e0-6e1a01b55dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:01.940315 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e12c2e8-e99b-4f5e-b3e0-6e1a01b55dc9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:01.941062 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e12c2e8-e99b-4f5e-b3e0-6e1a01b55dc9 None admin] RBAC: Adding query filter params (role.id=33bd60d3b0e94dcbbfe97e87f91a8d78, scope.project.id=105a2b33e5ee4cf88b11c48aebf329d7, user.id=8f1a8fbc56c44c829c8dd8013a782220) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:01.941374 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e12c2e8-e99b-4f5e-b3e0-6e1a01b55dc9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:01.942902 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e12c2e8-e99b-4f5e-b3e0-6e1a01b55dc9 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:01.949594 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 44/90] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:30:01 2022] GET /identity/v3/role_assignments?user.id=8f1a8fbc56c44c829c8dd8013a782220&scope.project.id=105a2b33e5ee4cf88b11c48aebf329d7&role.id=33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:01.950427 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e12c2e8-e99b-4f5e-b3e0-6e1a01b55dc9 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:01.958253 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:02.029812 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:02.034079 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46650280-bc89-4ae3-9f66-72945b53beff None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:02.072388 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6cb40-753d-4266-9098-7bc334f09127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:02.072757 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6cb40-753d-4266-9098-7bc334f09127 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:02.073047 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6cb40-753d-4266-9098-7bc334f09127 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:02.073328 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6cb40-753d-4266-9098-7bc334f09127 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:02.132873 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e12c2e8-e99b-4f5e-b3e0-6e1a01b55dc9 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:02.155155 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b6cb40-753d-4266-9098-7bc334f09127 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:02.156722 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b6cb40-753d-4266-9098-7bc334f09127 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:02.207750 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:02.226203 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c3404b-8f41-40d4-8a98-6db4a949828d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:02.227703 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3404b-8f41-40d4-8a98-6db4a949828d None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:02.228266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3404b-8f41-40d4-8a98-6db4a949828d None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:02.228767 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3404b-8f41-40d4-8a98-6db4a949828d None admin] PATH_INFO: `/v3/users/45f6f1fc05fd4bf590127b863244444f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:02.232681 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c3404b-8f41-40d4-8a98-6db4a949828d None admin] RBAC: Authorizing `identity:get_user(user_id=45f6f1fc05fd4bf590127b863244444f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:02.234413 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c3404b-8f41-40d4-8a98-6db4a949828d None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:02.236304 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 45/91] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:30:01 2022] GET /identity/v3/users/45f6f1fc05fd4bf590127b863244444f => generated 275 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:02.242308 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c3404b-8f41-40d4-8a98-6db4a949828d None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:02.260627 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 47/92] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:30:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:30:02.269234 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6cb40-753d-4266-9098-7bc334f09127 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:02.397995 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6cb40-753d-4266-9098-7bc334f09127 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:02.460892 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c3404b-8f41-40d4-8a98-6db4a949828d None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:02.484873 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:02.486510 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a5447-69e3-4614-8a14-586e09a92523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:02.487431 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a5447-69e3-4614-8a14-586e09a92523 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:02.487809 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a5447-69e3-4614-8a14-586e09a92523 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:02.488232 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a5447-69e3-4614-8a14-586e09a92523 None admin] PATH_INFO: `/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:02.491154 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5a5447-69e3-4614-8a14-586e09a92523 None admin] RBAC: Authorizing `identity:get_project(project_id=105a2b33e5ee4cf88b11c48aebf329d7)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:02.492577 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5a5447-69e3-4614-8a14-586e09a92523 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:02.494871 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 48/93] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:02 2022] GET /identity/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:02.500229 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0bbcd1-477f-49ee-93c7-7ea9ac5cbdd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:02.501347 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0bbcd1-477f-49ee-93c7-7ea9ac5cbdd9 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:02.501460 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a5447-69e3-4614-8a14-586e09a92523 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:02.501888 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0bbcd1-477f-49ee-93c7-7ea9ac5cbdd9 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:02.502304 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0bbcd1-477f-49ee-93c7-7ea9ac5cbdd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:02.532355 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:02.574844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0bbcd1-477f-49ee-93c7-7ea9ac5cbdd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:02.577105 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0bbcd1-477f-49ee-93c7-7ea9ac5cbdd9 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:02.706200 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a5447-69e3-4614-8a14-586e09a92523 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:02.728430 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 46/94] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:30:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:30:02.734382 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0bbcd1-477f-49ee-93c7-7ea9ac5cbdd9 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:02.771979 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:02.772163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f96363-6d59-490f-a585-dbbb6a552ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:02.772163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f96363-6d59-490f-a585-dbbb6a552ca4 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:02.772163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f96363-6d59-490f-a585-dbbb6a552ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:02.772163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f96363-6d59-490f-a585-dbbb6a552ca4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:02.772515 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f96363-6d59-490f-a585-dbbb6a552ca4 None admin] RBAC: Adding query filter params (role.id=3428ff8e92274de6817b7e8ee9a56d13, scope.project.id=105a2b33e5ee4cf88b11c48aebf329d7, user.id=45f6f1fc05fd4bf590127b863244444f) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:02.772589 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f96363-6d59-490f-a585-dbbb6a552ca4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:02.774782 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f96363-6d59-490f-a585-dbbb6a552ca4 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:02.786029 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 49/95] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:30:02 2022] GET /identity/v3/role_assignments?user.id=45f6f1fc05fd4bf590127b863244444f&scope.project.id=105a2b33e5ee4cf88b11c48aebf329d7&role.id=3428ff8e92274de6817b7e8ee9a56d13 => generated 259 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:02.855158 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0bbcd1-477f-49ee-93c7-7ea9ac5cbdd9 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:02.893697 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e36dedb-2056-4521-8664-0881fcdbdc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:02.894694 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36dedb-2056-4521-8664-0881fcdbdc32 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:02.895203 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36dedb-2056-4521-8664-0881fcdbdc32 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:02.895682 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36dedb-2056-4521-8664-0881fcdbdc32 None admin] PATH_INFO: `/v3/projects/f7d5d1f74f3448008365b515fc91db2c` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:02.897130 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e36dedb-2056-4521-8664-0881fcdbdc32 None admin] RBAC: Authorizing `identity:get_project(project_id=f7d5d1f74f3448008365b515fc91db2c)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:02.898836 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e36dedb-2056-4521-8664-0881fcdbdc32 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:02.900889 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 47/96] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:02 2022] GET /identity/v3/projects/f7d5d1f74f3448008365b515fc91db2c => generated 312 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:30:02.907764 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f96363-6d59-490f-a585-dbbb6a552ca4 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:03.008018 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:03.036202 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f96363-6d59-490f-a585-dbbb6a552ca4 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:03.086023 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f504d1-ba20-4619-90ca-00e79db9345b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:03.086641 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f504d1-ba20-4619-90ca-00e79db9345b None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:03.086925 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f504d1-ba20-4619-90ca-00e79db9345b None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:03.087202 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f504d1-ba20-4619-90ca-00e79db9345b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:03.087935 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f504d1-ba20-4619-90ca-00e79db9345b None admin] RBAC: Adding query filter params (role.id=1182317a465349478dde4e28cb7cf96f, scope.project.id=f7d5d1f74f3448008365b515fc91db2c, user.id=45f6f1fc05fd4bf590127b863244444f) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:03.088273 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f504d1-ba20-4619-90ca-00e79db9345b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:03.089735 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f504d1-ba20-4619-90ca-00e79db9345b None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:03.099874 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 50/97] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:30:02 2022] GET /identity/v3/role_assignments?user.id=45f6f1fc05fd4bf590127b863244444f&scope.project.id=f7d5d1f74f3448008365b515fc91db2c&role.id=1182317a465349478dde4e28cb7cf96f => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:04.323100 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daeaa11-bbc2-4305-93b5-287018b65881 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:04.323603 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daeaa11-bbc2-4305-93b5-287018b65881 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:04.323704 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daeaa11-bbc2-4305-93b5-287018b65881 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:04.325032 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 48/98] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:04.330900 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b74c60-8016-4894-9e6a-1ff011144121 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:04.331717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b74c60-8016-4894-9e6a-1ff011144121 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:04.332225 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b74c60-8016-4894-9e6a-1ff011144121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:04.356130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-43b74c60-8016-4894-9e6a-1ff011144121 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:04.357718 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-43b74c60-8016-4894-9e6a-1ff011144121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:04.394471 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ed67a8-9f6e-43e2-a33d-ce7a140f50c4 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:04.394769 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ed67a8-9f6e-43e2-a33d-ce7a140f50c4 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:04.395031 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ed67a8-9f6e-43e2-a33d-ce7a140f50c4 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:04.396415 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 49/99] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:04.401178 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf3365-3f51-48d1-86c7-00ebddbf1700 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:04.401463 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf3365-3f51-48d1-86c7-00ebddbf1700 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:04.401748 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf3365-3f51-48d1-86c7-00ebddbf1700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:04.420856 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-6daf3365-3f51-48d1-86c7-00ebddbf1700 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:04.422169 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-6daf3365-3f51-48d1-86c7-00ebddbf1700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:04.475603 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 51/100] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:04.530940 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 50/101] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 23:30:04.670897 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882691b2-ee06-43eb-817e-353305de244a None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:04.671489 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882691b2-ee06-43eb-817e-353305de244a None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:04.671955 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882691b2-ee06-43eb-817e-353305de244a None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:04.673580 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 52/102] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:04.678892 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885613f8-7835-498b-8c7c-02aefa21e106 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:04.679405 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885613f8-7835-498b-8c7c-02aefa21e106 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:04.679871 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885613f8-7835-498b-8c7c-02aefa21e106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:04.704076 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-885613f8-7835-498b-8c7c-02aefa21e106 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:04.705612 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-885613f8-7835-498b-8c7c-02aefa21e106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:04.729061 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:04.787613 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882691b2-ee06-43eb-817e-353305de244a None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:04.802091 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 51/103] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:04.885607 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885613f8-7835-498b-8c7c-02aefa21e106 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:04.920492 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:05.005458 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882691b2-ee06-43eb-817e-353305de244a None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:05.062612 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885613f8-7835-498b-8c7c-02aefa21e106 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:05.088649 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5151565-3420-45d8-9f16-b3182d5256a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:05.089296 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5151565-3420-45d8-9f16-b3182d5256a3 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:05.089587 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5151565-3420-45d8-9f16-b3182d5256a3 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:05.089867 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5151565-3420-45d8-9f16-b3182d5256a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:05.097529 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7c5050-9b50-4d40-b69a-3ca4f2e8b886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:05.098156 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c5050-9b50-4d40-b69a-3ca4f2e8b886 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:05.098433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c5050-9b50-4d40-b69a-3ca4f2e8b886 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:05.098702 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c5050-9b50-4d40-b69a-3ca4f2e8b886 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:05.109479 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7c5050-9b50-4d40-b69a-3ca4f2e8b886 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:05.114483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-de7c5050-9b50-4d40-b69a-3ca4f2e8b886 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 23:30:05.114483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:05.114483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:05.114483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:05.114483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:05.114483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:05.114483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:05.114483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:05.114483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:05.114483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:05.114483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:05.114483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:05.114483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:05.114483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:05.114483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:05.114483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:05.115343 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:05.115343 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:05.115343 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:05.115343 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:05.115343 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:05.115343 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:05.115343 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:05.115343 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:05.115343 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:05.115343 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:05.115343 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:05.115343 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:05.115343 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:05.115343 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:05.115343 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:05.115343 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:05.116172 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:05.116172 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:05.116172 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:05.116172 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:05.116172 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:05.116172 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:05.116172 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:05.116172 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:05.116172 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:05.116172 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:05.116172 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 23:30:05.116172 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:30:05.116172 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 52/104] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:30:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:30:05.117858 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2953eb-c33e-429b-b1d7-8072109ce152 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:05.117958 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2953eb-c33e-429b-b1d7-8072109ce152 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:05.118026 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5151565-3420-45d8-9f16-b3182d5256a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:05.118191 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2953eb-c33e-429b-b1d7-8072109ce152 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:05.123321 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 53/105] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:05.123435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-b5151565-3420-45d8-9f16-b3182d5256a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:30:05.123435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:05.123435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:05.123435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:05.123435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:05.123435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:05.123435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:05.123435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:05.123435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:05.123435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:05.123435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:05.123435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:05.123435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:05.123435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:05.123435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:05.123435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:05.124275 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:05.124275 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:05.124275 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:05.124275 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:05.124275 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:05.124275 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:05.124275 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:05.124275 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:05.124275 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:05.124275 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:05.124275 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:05.124275 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:05.124275 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:05.124275 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:05.124275 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:05.124275 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:05.126428 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:05.126428 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:05.126428 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:05.126428 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:05.126428 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:05.126428 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:05.126428 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:05.126428 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:05.126428 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:05.126428 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:05.126428 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:30:05.126428 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:30:05.127853 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2953eb-c33e-429b-b1d7-8072109ce152 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:05.128010 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 53/106] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 17 23:30:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:30:05.138972 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5151565-3420-45d8-9f16-b3182d5256a3 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:05.156311 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:05.235468 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2953eb-c33e-429b-b1d7-8072109ce152 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:05.270996 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5322758-9f61-4b85-9e98-47335b2843e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:05.271527 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5322758-9f61-4b85-9e98-47335b2843e2 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:05.271804 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5322758-9f61-4b85-9e98-47335b2843e2 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:05.272108 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5322758-9f61-4b85-9e98-47335b2843e2 None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:05.272779 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5322758-9f61-4b85-9e98-47335b2843e2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:05.273080 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5322758-9f61-4b85-9e98-47335b2843e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:05.274465 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5322758-9f61-4b85-9e98-47335b2843e2 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:05.283449 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 54/107] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 17 23:30:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:05.284759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5201f5-3da3-4dd8-bbc5-31c78040b901 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:05.285042 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5201f5-3da3-4dd8-bbc5-31c78040b901 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:05.285315 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5201f5-3da3-4dd8-bbc5-31c78040b901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:05.306586 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-bd5201f5-3da3-4dd8-bbc5-31c78040b901 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:05.307944 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-bd5201f5-3da3-4dd8-bbc5-31c78040b901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:05.328928 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5151565-3420-45d8-9f16-b3182d5256a3 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:05.365756 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e767ab-5aba-4e04-bde6-84bba7467bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:05.366490 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e767ab-5aba-4e04-bde6-84bba7467bed None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:05.366815 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e767ab-5aba-4e04-bde6-84bba7467bed None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:05.367089 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e767ab-5aba-4e04-bde6-84bba7467bed None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:05.367932 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e767ab-5aba-4e04-bde6-84bba7467bed None admin] RBAC: Adding query filter params (name=admin) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:05.368283 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e767ab-5aba-4e04-bde6-84bba7467bed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:05.369712 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e767ab-5aba-4e04-bde6-84bba7467bed None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:05.379042 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 54/108] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:30:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:05.379889 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e767ab-5aba-4e04-bde6-84bba7467bed None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:05.395609 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 55/109] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:05.397260 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:05.401090 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5201f5-3da3-4dd8-bbc5-31c78040b901 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:05.407875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:05.580636 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5201f5-3da3-4dd8-bbc5-31c78040b901 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:05.593569 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e767ab-5aba-4e04-bde6-84bba7467bed None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:05.616657 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083559b9-db9a-447f-8d2f-45f0f2da0ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:05.617630 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083559b9-db9a-447f-8d2f-45f0f2da0ba1 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:05.618140 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083559b9-db9a-447f-8d2f-45f0f2da0ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:05.618612 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083559b9-db9a-447f-8d2f-45f0f2da0ba1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:05.629393 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083559b9-db9a-447f-8d2f-45f0f2da0ba1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:05.634520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-083559b9-db9a-447f-8d2f-45f0f2da0ba1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 23:30:05.634520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:05.634520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:05.634520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:05.634520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:05.634520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:05.634520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:05.634520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:05.634520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:05.634520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:05.634520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:05.634520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:05.634520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:05.634520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:05.634520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:05.634520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:05.635358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:05.635358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:05.635358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:05.635358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:05.635358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:05.635358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:05.635358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:05.635358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:05.635358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:05.635358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:05.635358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:05.635358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:05.635358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:05.635358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:05.635358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:05.635358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:05.636516 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:05.636516 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:05.636516 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:05.636516 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:05.636516 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:05.636516 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:05.636516 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:05.636516 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:05.636516 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:05.636516 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:05.636516 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 23:30:05.636516 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:30:05.639303 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 56/110] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:30:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:05.640304 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083559b9-db9a-447f-8d2f-45f0f2da0ba1 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:05.677828 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab50d5fe-490c-47d8-996a-61ebd13e2f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:05.678052 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50d5fe-490c-47d8-996a-61ebd13e2f1f None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:05.678351 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50d5fe-490c-47d8-996a-61ebd13e2f1f None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:05.678636 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50d5fe-490c-47d8-996a-61ebd13e2f1f None admin] PATH_INFO: `/v3/users/75bac2f681594654bfb749088ae12b19` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:05.707601 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab50d5fe-490c-47d8-996a-61ebd13e2f1f None admin] RBAC: Authorizing `identity:get_user(user_id=75bac2f681594654bfb749088ae12b19)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:05.710092 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab50d5fe-490c-47d8-996a-61ebd13e2f1f None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:05.711913 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 55/111] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:30:05 2022] GET /identity/v3/users/75bac2f681594654bfb749088ae12b19 => generated 289 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:05.713574 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab50d5fe-490c-47d8-996a-61ebd13e2f1f None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:05.715302 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:05.768293 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083559b9-db9a-447f-8d2f-45f0f2da0ba1 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:05.776363 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:05.815345 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e5dae-df8a-4ef9-9437-959d44d655a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:05.816601 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e5dae-df8a-4ef9-9437-959d44d655a1 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:05.817317 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e5dae-df8a-4ef9-9437-959d44d655a1 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:05.818190 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e5dae-df8a-4ef9-9437-959d44d655a1 None admin] PATH_INFO: `/v3/users/8f1a8fbc56c44c829c8dd8013a782220` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:05.819842 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871e5dae-df8a-4ef9-9437-959d44d655a1 None admin] RBAC: Authorizing `identity:get_user(user_id=8f1a8fbc56c44c829c8dd8013a782220)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:05.822367 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871e5dae-df8a-4ef9-9437-959d44d655a1 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:05.824072 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab50d5fe-490c-47d8-996a-61ebd13e2f1f None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:05.824363 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 57/112] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:30:05 2022] GET /identity/v3/users/8f1a8fbc56c44c829c8dd8013a782220 => generated 247 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:05.825431 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e5dae-df8a-4ef9-9437-959d44d655a1 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:05.863070 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2432ce17-0d8b-4ede-9a6d-7b8df77750bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:05.863801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2432ce17-0d8b-4ede-9a6d-7b8df77750bf None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:05.864400 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2432ce17-0d8b-4ede-9a6d-7b8df77750bf None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:05.864610 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2432ce17-0d8b-4ede-9a6d-7b8df77750bf None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:05.865400 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2432ce17-0d8b-4ede-9a6d-7b8df77750bf None admin] RBAC: Adding query filter params (name=member) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:05.865800 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2432ce17-0d8b-4ede-9a6d-7b8df77750bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:05.867359 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2432ce17-0d8b-4ede-9a6d-7b8df77750bf None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:05.876998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 56/113] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 17 23:30:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:05.877832 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2432ce17-0d8b-4ede-9a6d-7b8df77750bf None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:05.942417 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e5dae-df8a-4ef9-9437-959d44d655a1 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:05.970156 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:05.976417 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aa7df6-09de-49d5-802a-810ed377fa19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:05.977042 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa7df6-09de-49d5-802a-810ed377fa19 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:05.977325 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa7df6-09de-49d5-802a-810ed377fa19 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:05.977593 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa7df6-09de-49d5-802a-810ed377fa19 None admin] PATH_INFO: `/v3/domains/default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:05.978710 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aa7df6-09de-49d5-802a-810ed377fa19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:05.980194 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aa7df6-09de-49d5-802a-810ed377fa19 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:05.981660 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2432ce17-0d8b-4ede-9a6d-7b8df77750bf None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:05.981771 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 58/114] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:30:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:05.982520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aa7df6-09de-49d5-802a-810ed377fa19 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:05.995617 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:06.017589 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df37bf3-a20d-45be-842a-ac072c296fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:06.018193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df37bf3-a20d-45be-842a-ac072c296fe1 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:06.018472 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df37bf3-a20d-45be-842a-ac072c296fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:06.018743 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df37bf3-a20d-45be-842a-ac072c296fe1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:06.080709 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aa7df6-09de-49d5-802a-810ed377fa19 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:06.090975 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df37bf3-a20d-45be-842a-ac072c296fe1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:06.093006 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df37bf3-a20d-45be-842a-ac072c296fe1 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:06.113846 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b66e02-ba4b-4460-8832-42e4108803c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:06.114411 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b66e02-ba4b-4460-8832-42e4108803c9 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:06.114688 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b66e02-ba4b-4460-8832-42e4108803c9 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:06.114961 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b66e02-ba4b-4460-8832-42e4108803c9 None admin] PATH_INFO: `/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:06.116137 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b66e02-ba4b-4460-8832-42e4108803c9 None admin] RBAC: Authorizing `identity:get_project(project_id=105a2b33e5ee4cf88b11c48aebf329d7)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:06.117570 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b66e02-ba4b-4460-8832-42e4108803c9 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:06.119187 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 59/115] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:05 2022] GET /identity/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7 => generated 298 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:06.120309 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42a385-64b9-4521-9bc8-8b24072a7bfb None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:06.120589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42a385-64b9-4521-9bc8-8b24072a7bfb None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:06.120862 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42a385-64b9-4521-9bc8-8b24072a7bfb None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:06.125953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 60/116] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:06.125953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf42a385-64b9-4521-9bc8-8b24072a7bfb None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:06.189083 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:06.192832 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 57/117] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:30:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:30:06.194216 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e458242-799c-476c-a141-87c8bbcceacc None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:06.194509 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e458242-799c-476c-a141-87c8bbcceacc None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:06.194784 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e458242-799c-476c-a141-87c8bbcceacc None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:06.199799 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 58/118] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:06.199799 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e458242-799c-476c-a141-87c8bbcceacc None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:06.210631 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:06.231038 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf42a385-64b9-4521-9bc8-8b24072a7bfb None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:06.265061 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ab705-55b4-4fc9-83f0-90017b224f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:06.265867 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ab705-55b4-4fc9-83f0-90017b224f37 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:06.266155 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ab705-55b4-4fc9-83f0-90017b224f37 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:06.266428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ab705-55b4-4fc9-83f0-90017b224f37 None admin] PATH_INFO: `/v3/users/45f6f1fc05fd4bf590127b863244444f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:06.267549 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ab705-55b4-4fc9-83f0-90017b224f37 None admin] RBAC: Authorizing `identity:get_user(user_id=45f6f1fc05fd4bf590127b863244444f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:06.269038 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ab705-55b4-4fc9-83f0-90017b224f37 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:06.270625 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 61/119] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:30:06 2022] GET /identity/v3/users/45f6f1fc05fd4bf590127b863244444f => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:06.273202 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ab705-55b4-4fc9-83f0-90017b224f37 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:06.300063 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e458242-799c-476c-a141-87c8bbcceacc None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:06.332755 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5a29fc-f61e-4461-9711-fbedd54deb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:06.333355 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a29fc-f61e-4461-9711-fbedd54deb70 None admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:06.333633 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a29fc-f61e-4461-9711-fbedd54deb70 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:06.333904 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a29fc-f61e-4461-9711-fbedd54deb70 None admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:06.334543 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5a29fc-f61e-4461-9711-fbedd54deb70 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:06.336850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5a29fc-f61e-4461-9711-fbedd54deb70 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:06.362885 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 59/120] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Aug 17 23:30:06 2022] POST /identity/v3/projects => generated 302 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:30:06.362885 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea01d5-064a-449a-9a19-79d01ee2ff49 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:06.362885 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea01d5-064a-449a-9a19-79d01ee2ff49 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:06.362885 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea01d5-064a-449a-9a19-79d01ee2ff49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:06.383477 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-13ea01d5-064a-449a-9a19-79d01ee2ff49 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:06.384706 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-13ea01d5-064a-449a-9a19-79d01ee2ff49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:06.390412 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ab705-55b4-4fc9-83f0-90017b224f37 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:06.430979 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808016cc-23c5-4f6c-b194-f7ec24920adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:06.431634 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808016cc-23c5-4f6c-b194-f7ec24920adb None admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:06.432720 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808016cc-23c5-4f6c-b194-f7ec24920adb None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:06.432720 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808016cc-23c5-4f6c-b194-f7ec24920adb None admin] PATH_INFO: `/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7/users/8f1a8fbc56c44c829c8dd8013a782220/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:06.434965 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808016cc-23c5-4f6c-b194-f7ec24920adb None admin] RBAC: Authorizing `identity:create_grant(project_id=105a2b33e5ee4cf88b11c48aebf329d7, user_id=8f1a8fbc56c44c829c8dd8013a782220, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:06.436632 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808016cc-23c5-4f6c-b194-f7ec24920adb None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:06.444703 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:06.459223 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 62/121] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 17 23:30:06 2022] PUT /identity/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7/users/8f1a8fbc56c44c829c8dd8013a782220/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:30:06.461141 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808016cc-23c5-4f6c-b194-f7ec24920adb None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:06.463516 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:06.478804 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 60/122] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:06.485842 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a1ccd-6892-4756-9c48-613bb2f11dba None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:06.486138 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a1ccd-6892-4756-9c48-613bb2f11dba None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:06.486419 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a1ccd-6892-4756-9c48-613bb2f11dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:06.535292 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-793a1ccd-6892-4756-9c48-613bb2f11dba None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:06.536593 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-793a1ccd-6892-4756-9c48-613bb2f11dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:06.616761 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808016cc-23c5-4f6c-b194-f7ec24920adb None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:06.638342 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 61/123] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:06.639130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793a1ccd-6892-4756-9c48-613bb2f11dba None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:06.677336 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ce5b76-8d54-46d2-a6c6-45c4d0f0fea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:06.678356 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce5b76-8d54-46d2-a6c6-45c4d0f0fea6 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:06.678741 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce5b76-8d54-46d2-a6c6-45c4d0f0fea6 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:06.679106 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce5b76-8d54-46d2-a6c6-45c4d0f0fea6 None admin] PATH_INFO: `/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:06.680424 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ce5b76-8d54-46d2-a6c6-45c4d0f0fea6 None admin] RBAC: Authorizing `identity:get_project(project_id=105a2b33e5ee4cf88b11c48aebf329d7)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:06.682045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ce5b76-8d54-46d2-a6c6-45c4d0f0fea6 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:06.684105 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 63/124] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:06 2022] GET /identity/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:06.692001 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ce5b76-8d54-46d2-a6c6-45c4d0f0fea6 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:06.790769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793a1ccd-6892-4756-9c48-613bb2f11dba None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:06.810489 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:06.838186 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:06.852283 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947146ba-272d-4555-9cc7-a3f9f427e82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:06.853295 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947146ba-272d-4555-9cc7-a3f9f427e82a None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:06.853769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947146ba-272d-4555-9cc7-a3f9f427e82a None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:06.854260 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947146ba-272d-4555-9cc7-a3f9f427e82a None admin] PATH_INFO: `/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:06.855664 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947146ba-272d-4555-9cc7-a3f9f427e82a None admin] RBAC: Authorizing `identity:get_project(project_id=105a2b33e5ee4cf88b11c48aebf329d7)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:06.857418 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947146ba-272d-4555-9cc7-a3f9f427e82a None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:06.859264 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 62/125] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:06 2022] GET /identity/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:06.861975 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ce5b76-8d54-46d2-a6c6-45c4d0f0fea6 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:06.864929 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947146ba-272d-4555-9cc7-a3f9f427e82a None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:06.924739 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ccb19-3df6-40c0-af8d-ed5d9482daa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:06.925665 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ccb19-3df6-40c0-af8d-ed5d9482daa0 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:06.925952 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ccb19-3df6-40c0-af8d-ed5d9482daa0 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:06.926390 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ccb19-3df6-40c0-af8d-ed5d9482daa0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:06.927250 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ccb19-3df6-40c0-af8d-ed5d9482daa0 None admin] RBAC: Adding query filter params (role.id=df3442912b3d43f0a36ebb936b3cb102, scope.project.id=105a2b33e5ee4cf88b11c48aebf329d7, user.id=75bac2f681594654bfb749088ae12b19) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:06.927568 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ccb19-3df6-40c0-af8d-ed5d9482daa0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:06.929579 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ccb19-3df6-40c0-af8d-ed5d9482daa0 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:06.941176 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 64/126] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:30:06 2022] GET /identity/v3/role_assignments?user.id=75bac2f681594654bfb749088ae12b19&scope.project.id=105a2b33e5ee4cf88b11c48aebf329d7&role.id=df3442912b3d43f0a36ebb936b3cb102 => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:30:06.951925 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ccb19-3df6-40c0-af8d-ed5d9482daa0 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:07.057427 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947146ba-272d-4555-9cc7-a3f9f427e82a None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:07.130699 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548d186-afb4-420a-9e86-9a92266f47a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:07.131712 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548d186-afb4-420a-9e86-9a92266f47a5 None admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:07.132939 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548d186-afb4-420a-9e86-9a92266f47a5 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:07.133574 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548d186-afb4-420a-9e86-9a92266f47a5 None admin] PATH_INFO: `/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7/users/45f6f1fc05fd4bf590127b863244444f/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:07.136803 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f548d186-afb4-420a-9e86-9a92266f47a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=105a2b33e5ee4cf88b11c48aebf329d7, user_id=45f6f1fc05fd4bf590127b863244444f, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:07.140414 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f548d186-afb4-420a-9e86-9a92266f47a5 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:07.163474 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 63/127] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 17 23:30:06 2022] PUT /identity/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7/users/45f6f1fc05fd4bf590127b863244444f/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:30:07.164497 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548d186-afb4-420a-9e86-9a92266f47a5 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:07.170168 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ccb19-3df6-40c0-af8d-ed5d9482daa0 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:07.208734 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:07.222773 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59b598-82bc-4041-b002-daddb29d4a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:07.223724 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59b598-82bc-4041-b002-daddb29d4a76 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:07.224451 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59b598-82bc-4041-b002-daddb29d4a76 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:07.224999 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59b598-82bc-4041-b002-daddb29d4a76 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:07.241760 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:07.242924 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b59b598-82bc-4041-b002-daddb29d4a76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:07.253593 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-6b59b598-82bc-4041-b002-daddb29d4a76 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 23:30:07.253593 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:07.253593 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:07.253593 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:07.253593 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:07.253593 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:07.253593 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:07.253593 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:07.253593 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:07.253593 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:07.253593 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:07.253593 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:07.253593 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:07.253593 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:07.253593 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:07.253593 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:07.254730 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:07.254730 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:07.254730 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:07.254730 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:07.254730 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:07.254730 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:07.254730 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:07.254730 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:07.254730 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:07.254730 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:07.254730 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:07.254730 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:07.254730 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:07.254730 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:07.254730 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:07.254730 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:07.255677 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:07.255677 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:07.255677 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:07.255677 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:07.255677 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:07.255677 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:07.255677 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:07.255677 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:07.255677 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:07.255677 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:07.255677 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 23:30:07.255677 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:30:07.258456 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 65/128] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 17 23:30:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:07.264774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59b598-82bc-4041-b002-daddb29d4a76 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:07.302385 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548d186-afb4-420a-9e86-9a92266f47a5 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:07.340032 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becdcf36-618b-4023-8789-1558f99bda6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:07.340814 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becdcf36-618b-4023-8789-1558f99bda6d None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:07.341803 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becdcf36-618b-4023-8789-1558f99bda6d None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:07.342106 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becdcf36-618b-4023-8789-1558f99bda6d None admin] PATH_INFO: `/v3/roles/member` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:07.353556 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becdcf36-618b-4023-8789-1558f99bda6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:07.358518 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-becdcf36-618b-4023-8789-1558f99bda6d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 23:30:07.358518 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:07.358518 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:07.358518 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:07.358518 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:07.358518 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:07.358518 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:07.358518 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:07.358518 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:07.358518 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:07.358518 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:07.358518 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:07.358518 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:07.358518 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:07.358518 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:07.358518 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:07.359442 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:07.359442 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:07.359442 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:07.359442 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:07.359442 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:07.359442 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:07.359442 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:07.359442 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:07.359442 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:07.359442 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:07.359442 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:07.359442 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:07.359442 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:07.359442 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:07.359442 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:07.359442 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:07.360244 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:07.360244 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:07.360244 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:07.360244 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:07.360244 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:07.360244 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:07.360244 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:07.360244 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:07.360244 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:07.360244 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:07.360244 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 23:30:07.360244 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:30:07.360244 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 64/129] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:30:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:07.365931 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becdcf36-618b-4023-8789-1558f99bda6d None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:07.381531 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59b598-82bc-4041-b002-daddb29d4a76 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:07.419862 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735e249-ea14-45ae-9ae9-3823578f6422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:07.420799 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735e249-ea14-45ae-9ae9-3823578f6422 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:07.421288 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735e249-ea14-45ae-9ae9-3823578f6422 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:07.421757 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735e249-ea14-45ae-9ae9-3823578f6422 None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:07.422635 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1735e249-ea14-45ae-9ae9-3823578f6422 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:07.423163 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1735e249-ea14-45ae-9ae9-3823578f6422 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:07.425079 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1735e249-ea14-45ae-9ae9-3823578f6422 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:07.434556 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 66/130] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:30:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:30:07.441381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735e249-ea14-45ae-9ae9-3823578f6422 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:07.485993 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becdcf36-618b-4023-8789-1558f99bda6d None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:07.498348 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:07.500664 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:07.524504 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40bd91-b996-46fd-a3be-d8cb358ca2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:07.525108 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40bd91-b996-46fd-a3be-d8cb358ca2ac None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:07.525390 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40bd91-b996-46fd-a3be-d8cb358ca2ac None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:07.525705 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40bd91-b996-46fd-a3be-d8cb358ca2ac None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:07.526381 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe40bd91-b996-46fd-a3be-d8cb358ca2ac None admin] RBAC: Adding query filter params (name=member) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:07.526681 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe40bd91-b996-46fd-a3be-d8cb358ca2ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:07.529198 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe40bd91-b996-46fd-a3be-d8cb358ca2ac None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:07.538874 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 65/131] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 17 23:30:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:30:07.544119 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40bd91-b996-46fd-a3be-d8cb358ca2ac None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:07.558413 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735e249-ea14-45ae-9ae9-3823578f6422 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:07.595802 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37202edd-8677-4caa-ad11-a6e05a110b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:07.596800 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37202edd-8677-4caa-ad11-a6e05a110b58 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:07.598603 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37202edd-8677-4caa-ad11-a6e05a110b58 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:07.599098 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37202edd-8677-4caa-ad11-a6e05a110b58 None admin] PATH_INFO: `/v3/users/45f6f1fc05fd4bf590127b863244444f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:07.600483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37202edd-8677-4caa-ad11-a6e05a110b58 None admin] RBAC: Authorizing `identity:get_user(user_id=45f6f1fc05fd4bf590127b863244444f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:07.602235 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37202edd-8677-4caa-ad11-a6e05a110b58 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:07.604331 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 67/132] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:30:07 2022] GET /identity/v3/users/45f6f1fc05fd4bf590127b863244444f => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:07.610473 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37202edd-8677-4caa-ad11-a6e05a110b58 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:07.650414 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40bd91-b996-46fd-a3be-d8cb358ca2ac None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:07.686974 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f2d6b-31cc-4981-bb8e-2b0e42c50e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:07.687538 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f2d6b-31cc-4981-bb8e-2b0e42c50e27 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:07.687826 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f2d6b-31cc-4981-bb8e-2b0e42c50e27 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:07.688141 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f2d6b-31cc-4981-bb8e-2b0e42c50e27 None admin] PATH_INFO: `/v3/users/45f6f1fc05fd4bf590127b863244444f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:07.689631 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3f2d6b-31cc-4981-bb8e-2b0e42c50e27 None admin] RBAC: Authorizing `identity:get_user(user_id=45f6f1fc05fd4bf590127b863244444f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:07.690847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3f2d6b-31cc-4981-bb8e-2b0e42c50e27 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:07.692624 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 66/133] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:30:07 2022] GET /identity/v3/users/45f6f1fc05fd4bf590127b863244444f => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:30:07.698521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f2d6b-31cc-4981-bb8e-2b0e42c50e27 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:07.725476 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37202edd-8677-4caa-ad11-a6e05a110b58 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:07.744155 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:07.744708 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:07.766191 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a15b5-867a-4ed3-96ba-83641066e45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:07.766836 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a15b5-867a-4ed3-96ba-83641066e45b None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:07.767119 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a15b5-867a-4ed3-96ba-83641066e45b None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:07.767391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a15b5-867a-4ed3-96ba-83641066e45b None admin] PATH_INFO: `/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:07.768566 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265a15b5-867a-4ed3-96ba-83641066e45b None admin] RBAC: Authorizing `identity:get_project(project_id=105a2b33e5ee4cf88b11c48aebf329d7)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:07.770030 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265a15b5-867a-4ed3-96ba-83641066e45b None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:07.771677 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 68/134] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:07 2022] GET /identity/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:30:07.778029 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a15b5-867a-4ed3-96ba-83641066e45b None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:07.809642 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f2d6b-31cc-4981-bb8e-2b0e42c50e27 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:07.845821 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4717775-a042-4d39-81a9-435804a93bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:07.846827 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4717775-a042-4d39-81a9-435804a93bef None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:07.847364 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4717775-a042-4d39-81a9-435804a93bef None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:07.847829 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4717775-a042-4d39-81a9-435804a93bef None admin] PATH_INFO: `/v3/projects/f7d5d1f74f3448008365b515fc91db2c` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:07.849314 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4717775-a042-4d39-81a9-435804a93bef None admin] RBAC: Authorizing `identity:get_project(project_id=f7d5d1f74f3448008365b515fc91db2c)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:07.850995 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4717775-a042-4d39-81a9-435804a93bef None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:07.852900 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 67/135] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:07 2022] GET /identity/v3/projects/f7d5d1f74f3448008365b515fc91db2c => generated 312 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:07.860166 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4717775-a042-4d39-81a9-435804a93bef None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:07.884613 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a15b5-867a-4ed3-96ba-83641066e45b None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:07.921175 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e1a02-7669-49ac-ac51-4bb3ebf0d050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:07.922000 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e1a02-7669-49ac-ac51-4bb3ebf0d050 None admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:07.922106 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e1a02-7669-49ac-ac51-4bb3ebf0d050 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:07.922360 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e1a02-7669-49ac-ac51-4bb3ebf0d050 None admin] PATH_INFO: `/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7/users/45f6f1fc05fd4bf590127b863244444f/roles/3428ff8e92274de6817b7e8ee9a56d13` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:07.925000 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e1a02-7669-49ac-ac51-4bb3ebf0d050 None admin] RBAC: Authorizing `identity:create_grant(project_id=105a2b33e5ee4cf88b11c48aebf329d7, user_id=45f6f1fc05fd4bf590127b863244444f, role_id=3428ff8e92274de6817b7e8ee9a56d13)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:07.927591 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e1a02-7669-49ac-ac51-4bb3ebf0d050 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:07.950187 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 69/136] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 17 23:30:07 2022] PUT /identity/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7/users/45f6f1fc05fd4bf590127b863244444f/roles/3428ff8e92274de6817b7e8ee9a56d13 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:30:07.988345 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4717775-a042-4d39-81a9-435804a93bef None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:08.000836 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:08.027314 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58d014-ad01-425d-9fca-e4636111ff73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:08.029251 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58d014-ad01-425d-9fca-e4636111ff73 None admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:08.029727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58d014-ad01-425d-9fca-e4636111ff73 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:08.030169 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58d014-ad01-425d-9fca-e4636111ff73 None admin] PATH_INFO: `/v3/projects/f7d5d1f74f3448008365b515fc91db2c/users/45f6f1fc05fd4bf590127b863244444f/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:08.032469 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c58d014-ad01-425d-9fca-e4636111ff73 None admin] RBAC: Authorizing `identity:create_grant(project_id=f7d5d1f74f3448008365b515fc91db2c, user_id=45f6f1fc05fd4bf590127b863244444f, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:08.034115 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c58d014-ad01-425d-9fca-e4636111ff73 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:08.053009 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 68/137] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 17 23:30:07 2022] PUT /identity/v3/projects/f7d5d1f74f3448008365b515fc91db2c/users/45f6f1fc05fd4bf590127b863244444f/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:30:09.604814 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0f33d-845b-4070-94f2-a77c94ab686a None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:09.605313 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0f33d-845b-4070-94f2-a77c94ab686a None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:09.605386 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0f33d-845b-4070-94f2-a77c94ab686a None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:09.607564 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 70/138] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:09.615059 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae14b11-5527-4d83-befe-6e01f473b3c2 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:09.615870 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae14b11-5527-4d83-befe-6e01f473b3c2 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:09.616394 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae14b11-5527-4d83-befe-6e01f473b3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:09.635976 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-cae14b11-5527-4d83-befe-6e01f473b3c2 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:09.637531 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-cae14b11-5527-4d83-befe-6e01f473b3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:09.707150 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 69/139] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:09.920578 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f82db-c7df-4bc0-a7f3-88381e12613b None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:09.920943 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f82db-c7df-4bc0-a7f3-88381e12613b None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:09.921132 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f82db-c7df-4bc0-a7f3-88381e12613b None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:09.922491 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 71/140] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:09.927617 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be811091-0ccf-4622-8b29-fcb7d3d0545e None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:09.928148 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be811091-0ccf-4622-8b29-fcb7d3d0545e None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:09.928617 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be811091-0ccf-4622-8b29-fcb7d3d0545e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:09.940876 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8dc956-519e-4fa4-98a4-207cc6e61f63 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:09.941461 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8dc956-519e-4fa4-98a4-207cc6e61f63 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:09.941925 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8dc956-519e-4fa4-98a4-207cc6e61f63 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:09.943420 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 72/141] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:09.948366 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fa33a-af43-40cc-a489-8497e69118ed None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:09.948872 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fa33a-af43-40cc-a489-8497e69118ed None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:09.949354 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fa33a-af43-40cc-a489-8497e69118ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:09.965186 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-be811091-0ccf-4622-8b29-fcb7d3d0545e None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:09.967143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-be811091-0ccf-4622-8b29-fcb7d3d0545e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:09.978205 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-ff9fa33a-af43-40cc-a489-8497e69118ed None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:09.979613 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-ff9fa33a-af43-40cc-a489-8497e69118ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:10.059470 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:10.106840 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 70/142] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:10.113830 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 73/143] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:10.147037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be811091-0ccf-4622-8b29-fcb7d3d0545e None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:10.179862 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:10.254667 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be811091-0ccf-4622-8b29-fcb7d3d0545e None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:10.290431 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d43b491-597f-4ac5-a1c8-c9e4c8fc78f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:10.291065 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43b491-597f-4ac5-a1c8-c9e4c8fc78f0 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:10.291347 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43b491-597f-4ac5-a1c8-c9e4c8fc78f0 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:10.291616 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43b491-597f-4ac5-a1c8-c9e4c8fc78f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:10.301967 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d43b491-597f-4ac5-a1c8-c9e4c8fc78f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:10.306832 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-7d43b491-597f-4ac5-a1c8-c9e4c8fc78f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:30:10.306832 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:10.306832 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:10.306832 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:10.306832 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:10.306832 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:10.306832 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:10.306832 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:10.306832 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:10.306832 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:10.306832 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:10.306832 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:10.306832 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:10.306832 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:10.306832 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:10.306832 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:10.307663 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:10.307663 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:10.307663 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:10.307663 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:10.307663 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:10.307663 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:10.307663 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:10.307663 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:10.307663 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:10.307663 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:10.307663 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:10.307663 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:10.307663 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:10.307663 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:10.307663 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:10.307663 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:10.308482 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:10.308482 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:10.308482 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:10.308482 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:10.308482 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:10.308482 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:10.308482 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:10.308482 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:10.308482 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:10.308482 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:10.308482 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:30:10.308482 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:30:10.308482 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 71/144] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 17 23:30:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:10.321420 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9fa33a-af43-40cc-a489-8497e69118ed None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:10.444865 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9fa33a-af43-40cc-a489-8497e69118ed None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:10.476394 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d43b491-597f-4ac5-a1c8-c9e4c8fc78f0 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:10.505300 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043b466-67bf-4036-876f-cf7617113b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:10.505850 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043b466-67bf-4036-876f-cf7617113b3e None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:10.506134 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043b466-67bf-4036-876f-cf7617113b3e None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:10.506415 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043b466-67bf-4036-876f-cf7617113b3e None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:10.507093 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6043b466-67bf-4036-876f-cf7617113b3e None admin] RBAC: Adding query filter params (name=admin) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:10.507399 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6043b466-67bf-4036-876f-cf7617113b3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:10.508830 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6043b466-67bf-4036-876f-cf7617113b3e None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:10.527346 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 74/145] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:30:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:10.536508 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043b466-67bf-4036-876f-cf7617113b3e None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:10.581390 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:10.591998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d43b491-597f-4ac5-a1c8-c9e4c8fc78f0 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:10.620323 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:10.657884 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed1c997-4ebb-4161-bde4-98db4e4d106e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:10.659261 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed1c997-4ebb-4161-bde4-98db4e4d106e None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:10.659538 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed1c997-4ebb-4161-bde4-98db4e4d106e None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:10.659807 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed1c997-4ebb-4161-bde4-98db4e4d106e None admin] PATH_INFO: `/v3/domains/default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:10.661598 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed1c997-4ebb-4161-bde4-98db4e4d106e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:10.664406 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed1c997-4ebb-4161-bde4-98db4e4d106e None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:10.665998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 72/146] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:30:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:30:10.666754 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed1c997-4ebb-4161-bde4-98db4e4d106e None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:10.742870 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043b466-67bf-4036-876f-cf7617113b3e None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:10.788860 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa122903-bdc9-4bf1-91bd-cc3a1c518510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:10.789419 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa122903-bdc9-4bf1-91bd-cc3a1c518510 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:10.789706 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa122903-bdc9-4bf1-91bd-cc3a1c518510 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:10.789982 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa122903-bdc9-4bf1-91bd-cc3a1c518510 None admin] PATH_INFO: `/v3/users/8f1a8fbc56c44c829c8dd8013a782220` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:10.791256 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa122903-bdc9-4bf1-91bd-cc3a1c518510 None admin] RBAC: Authorizing `identity:get_user(user_id=8f1a8fbc56c44c829c8dd8013a782220)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:10.792777 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa122903-bdc9-4bf1-91bd-cc3a1c518510 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:10.794610 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 75/147] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:30:10 2022] GET /identity/v3/users/8f1a8fbc56c44c829c8dd8013a782220 => generated 247 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:30:10.795356 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa122903-bdc9-4bf1-91bd-cc3a1c518510 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:10.807612 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed1c997-4ebb-4161-bde4-98db4e4d106e None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:10.843337 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f9272-d11a-488b-a6e0-bb2abbb6ed88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:10.844029 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f9272-d11a-488b-a6e0-bb2abbb6ed88 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:10.844310 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f9272-d11a-488b-a6e0-bb2abbb6ed88 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:10.844577 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f9272-d11a-488b-a6e0-bb2abbb6ed88 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:10.855163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f9272-d11a-488b-a6e0-bb2abbb6ed88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:10.860124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-d35f9272-d11a-488b-a6e0-bb2abbb6ed88 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 23:30:10.860124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:10.860124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:10.860124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:10.860124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:10.860124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:10.860124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:10.860124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:10.860124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:10.860124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:10.860124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:10.860124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:10.860124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:10.860124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:10.860124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:10.860124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:10.860972 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:10.860972 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:10.860972 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:10.860972 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:10.860972 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:10.860972 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:10.860972 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:10.860972 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:10.860972 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:10.860972 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:10.860972 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:10.860972 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:10.860972 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:10.860972 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:10.860972 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:10.860972 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:10.861726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:10.861726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:10.861726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:10.861726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:10.861726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:10.861726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:10.861726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:10.861726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:10.861726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:10.861726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:10.861726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 23:30:10.861726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:30:10.861726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 73/148] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:30:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:10.862822 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148ae80-3846-4218-ad39-4a358097a69f None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:10.862822 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148ae80-3846-4218-ad39-4a358097a69f None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:10.867990 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148ae80-3846-4218-ad39-4a358097a69f None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:10.867990 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 74/149] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:10.867990 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148ae80-3846-4218-ad39-4a358097a69f None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:10.884866 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:10.918142 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa122903-bdc9-4bf1-91bd-cc3a1c518510 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:10.920726 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:10.957537 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24430805-d198-4156-8e7f-bddfc8e517bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:10.958148 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24430805-d198-4156-8e7f-bddfc8e517bd None admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:10.958427 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24430805-d198-4156-8e7f-bddfc8e517bd None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:10.958699 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24430805-d198-4156-8e7f-bddfc8e517bd None admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:10.959342 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24430805-d198-4156-8e7f-bddfc8e517bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:10.960842 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24430805-d198-4156-8e7f-bddfc8e517bd None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:10.974320 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148ae80-3846-4218-ad39-4a358097a69f None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:10.984345 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 76/150] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Aug 17 23:30:10 2022] POST /identity/v3/users => generated 283 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:30:10.985307 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24430805-d198-4156-8e7f-bddfc8e517bd None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:11.020408 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489daebb-184c-4352-917f-ea4b84c730b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:11.021610 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489daebb-184c-4352-917f-ea4b84c730b2 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:11.022619 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489daebb-184c-4352-917f-ea4b84c730b2 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:11.023035 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489daebb-184c-4352-917f-ea4b84c730b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:11.114360 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489daebb-184c-4352-917f-ea4b84c730b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:11.116170 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489daebb-184c-4352-917f-ea4b84c730b2 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:11.119725 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24430805-d198-4156-8e7f-bddfc8e517bd None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:11.139470 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:11.159591 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ce728-4bb5-4cd6-b90e-c2f5d6bdb865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:11.160551 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ce728-4bb5-4cd6-b90e-c2f5d6bdb865 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:11.160828 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ce728-4bb5-4cd6-b90e-c2f5d6bdb865 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:11.161096 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ce728-4bb5-4cd6-b90e-c2f5d6bdb865 None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:11.161769 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ce728-4bb5-4cd6-b90e-c2f5d6bdb865 None admin] RBAC: Adding query filter params (name=reader) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:11.162066 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ce728-4bb5-4cd6-b90e-c2f5d6bdb865 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:11.163450 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ce728-4bb5-4cd6-b90e-c2f5d6bdb865 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:11.173396 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 77/151] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 17 23:30:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:11.177920 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d1dec-1284-48bd-843f-a7277ab2bcd8 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:11.177920 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d1dec-1284-48bd-843f-a7277ab2bcd8 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:11.177920 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d1dec-1284-48bd-843f-a7277ab2bcd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:11.195213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-ba8d1dec-1284-48bd-843f-a7277ab2bcd8 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:11.196433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-ba8d1dec-1284-48bd-843f-a7277ab2bcd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:11.221572 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:11.246197 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 75/152] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:30:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:30:11.251146 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aceafd0-556f-4d00-9491-21be6f57475d None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:11.251146 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aceafd0-556f-4d00-9491-21be6f57475d None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:11.251146 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aceafd0-556f-4d00-9491-21be6f57475d None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:11.251146 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 76/153] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:11.251146 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba8e83-5ea8-4a1c-9b4b-cbddaa602fc5 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:11.251502 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba8e83-5ea8-4a1c-9b4b-cbddaa602fc5 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:11.251567 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba8e83-5ea8-4a1c-9b4b-cbddaa602fc5 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:11.252799 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 77/154] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:11.253504 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba8e83-5ea8-4a1c-9b4b-cbddaa602fc5 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:11.287770 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 78/155] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:11.288782 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d1dec-1284-48bd-843f-a7277ab2bcd8 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:11.397020 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d1dec-1284-48bd-843f-a7277ab2bcd8 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:11.404690 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba8e83-5ea8-4a1c-9b4b-cbddaa602fc5 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:11.430425 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:11.447334 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8508863-f5d5-41db-b68c-904eff1151df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:11.448007 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8508863-f5d5-41db-b68c-904eff1151df None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:11.448295 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8508863-f5d5-41db-b68c-904eff1151df None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:11.448564 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8508863-f5d5-41db-b68c-904eff1151df None admin] PATH_INFO: `/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:11.449848 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8508863-f5d5-41db-b68c-904eff1151df None admin] RBAC: Authorizing `identity:get_project(project_id=105a2b33e5ee4cf88b11c48aebf329d7)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:11.451254 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8508863-f5d5-41db-b68c-904eff1151df None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:11.464010 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 79/156] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:11 2022] GET /identity/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:11.464010 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526f8ba-2cae-476b-9bb2-bbf5d077f8fd None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:11.464010 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526f8ba-2cae-476b-9bb2-bbf5d077f8fd None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:11.464010 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526f8ba-2cae-476b-9bb2-bbf5d077f8fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:11.464362 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e90af-9225-4a02-961e-103ffcc035e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:11.464362 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e90af-9225-4a02-961e-103ffcc035e0 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:11.464362 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e90af-9225-4a02-961e-103ffcc035e0 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:11.464362 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e90af-9225-4a02-961e-103ffcc035e0 None admin] PATH_INFO: `/v3/users/75bac2f681594654bfb749088ae12b19` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:11.464604 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8e90af-9225-4a02-961e-103ffcc035e0 None admin] RBAC: Authorizing `identity:get_user(user_id=75bac2f681594654bfb749088ae12b19)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:11.464796 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8e90af-9225-4a02-961e-103ffcc035e0 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:11.467163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 78/157] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:30:11 2022] GET /identity/v3/users/75bac2f681594654bfb749088ae12b19 => generated 289 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:11.468503 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624c583-a9cc-439c-a771-8edb3749759a None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:11.468933 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624c583-a9cc-439c-a771-8edb3749759a None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:11.469306 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624c583-a9cc-439c-a771-8edb3749759a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:11.485527 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-8526f8ba-2cae-476b-9bb2-bbf5d077f8fd None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:11.486707 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-8526f8ba-2cae-476b-9bb2-bbf5d077f8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:11.501101 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-1624c583-a9cc-439c-a771-8edb3749759a None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:11.502440 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:11.503766 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-1624c583-a9cc-439c-a771-8edb3749759a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:11.563398 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 80/158] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:11.572301 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8526f8ba-2cae-476b-9bb2-bbf5d077f8fd None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:11.602942 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 79/159] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:11.609810 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624c583-a9cc-439c-a771-8edb3749759a None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:11.728740 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624c583-a9cc-439c-a771-8edb3749759a None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:11.743983 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8526f8ba-2cae-476b-9bb2-bbf5d077f8fd None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:11.748924 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:11.771309 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc0f344-c845-401a-9d03-59898d96fdc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:11.772538 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0f344-c845-401a-9d03-59898d96fdc9 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:11.772952 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0f344-c845-401a-9d03-59898d96fdc9 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:11.773231 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0f344-c845-401a-9d03-59898d96fdc9 None admin] PATH_INFO: `/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:11.774501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc0f344-c845-401a-9d03-59898d96fdc9 None admin] RBAC: Authorizing `identity:get_project(project_id=105a2b33e5ee4cf88b11c48aebf329d7)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:11.776228 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc0f344-c845-401a-9d03-59898d96fdc9 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:11.779465 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 80/160] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:11 2022] GET /identity/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:11.780350 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc0f344-c845-401a-9d03-59898d96fdc9 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:11.816782 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28907be-dcdc-44ed-a3eb-f040b179126a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:11.817515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28907be-dcdc-44ed-a3eb-f040b179126a None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:11.817795 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28907be-dcdc-44ed-a3eb-f040b179126a None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:11.818067 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28907be-dcdc-44ed-a3eb-f040b179126a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:11.818802 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28907be-dcdc-44ed-a3eb-f040b179126a None admin] RBAC: Adding query filter params (role.id=33bd60d3b0e94dcbbfe97e87f91a8d78, scope.project.id=105a2b33e5ee4cf88b11c48aebf329d7, user.id=8f1a8fbc56c44c829c8dd8013a782220) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:11.819109 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28907be-dcdc-44ed-a3eb-f040b179126a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:11.820829 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28907be-dcdc-44ed-a3eb-f040b179126a None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:11.833494 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 81/161] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:30:11 2022] GET /identity/v3/role_assignments?user.id=8f1a8fbc56c44c829c8dd8013a782220&scope.project.id=105a2b33e5ee4cf88b11c48aebf329d7&role.id=33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 608 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:11.834397 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28907be-dcdc-44ed-a3eb-f040b179126a None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:11.865583 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:11.909484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc0f344-c845-401a-9d03-59898d96fdc9 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:11.953799 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd78b709-d342-4809-9c94-52f981313bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:11.954948 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78b709-d342-4809-9c94-52f981313bc0 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:11.955237 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78b709-d342-4809-9c94-52f981313bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:11.955504 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78b709-d342-4809-9c94-52f981313bc0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:11.975122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd78b709-d342-4809-9c94-52f981313bc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:11.980118 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-fd78b709-d342-4809-9c94-52f981313bc0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 23:30:11.980118 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:11.980118 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:11.980118 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:11.980118 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:11.980118 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:11.980118 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:11.980118 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:11.980118 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:11.980118 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:11.980118 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:11.980118 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:11.980118 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:11.980118 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:11.980118 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:11.980118 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:11.982743 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:11.982743 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:11.982743 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:11.982743 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:11.982743 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:11.982743 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:11.982743 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:11.982743 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:11.982743 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:11.982743 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:11.982743 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:11.982743 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:11.982743 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:11.982743 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:11.982743 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:11.982743 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:11.984643 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:11.984643 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:11.984643 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:11.984643 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:11.984643 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:11.984643 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:11.984643 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:11.984643 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:11.984643 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:11.984643 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:11.984643 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 23:30:11.984643 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:30:11.984643 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 81/162] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:30:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:11.984643 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd78b709-d342-4809-9c94-52f981313bc0 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:12.053972 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:12.086033 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28907be-dcdc-44ed-a3eb-f040b179126a None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:12.104539 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd78b709-d342-4809-9c94-52f981313bc0 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:12.146549 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457f26a7-a18f-4c9e-a439-4a5511337c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:12.147216 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f26a7-a18f-4c9e-a439-4a5511337c5e None admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:12.147498 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f26a7-a18f-4c9e-a439-4a5511337c5e None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:12.147723 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab6116-0020-47ef-9dda-608d984d8ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:12.147833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f26a7-a18f-4c9e-a439-4a5511337c5e None admin] PATH_INFO: `/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7/users/75bac2f681594654bfb749088ae12b19/roles/df3442912b3d43f0a36ebb936b3cb102` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:12.148410 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab6116-0020-47ef-9dda-608d984d8ad3 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:12.149176 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab6116-0020-47ef-9dda-608d984d8ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:12.149520 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab6116-0020-47ef-9dda-608d984d8ad3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:12.149800 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457f26a7-a18f-4c9e-a439-4a5511337c5e None admin] RBAC: Authorizing `identity:create_grant(project_id=105a2b33e5ee4cf88b11c48aebf329d7, user_id=75bac2f681594654bfb749088ae12b19, role_id=df3442912b3d43f0a36ebb936b3cb102)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:12.151311 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457f26a7-a18f-4c9e-a439-4a5511337c5e None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:12.163136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab6116-0020-47ef-9dda-608d984d8ad3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:12.167306 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 82/163] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 17 23:30:11 2022] PUT /identity/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7/users/75bac2f681594654bfb749088ae12b19/roles/df3442912b3d43f0a36ebb936b3cb102 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:30:12.168233 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457f26a7-a18f-4c9e-a439-4a5511337c5e None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:12.173231 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-d1ab6116-0020-47ef-9dda-608d984d8ad3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 23:30:12.173231 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:12.173231 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:12.173231 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:12.173231 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:12.173231 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:12.173231 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:12.173231 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:12.173231 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:12.173231 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:12.173231 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:12.173231 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:12.173231 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:12.173231 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:12.173231 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:12.173231 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:12.174143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:12.174143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:12.174143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:12.174143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:12.174143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:12.174143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:12.174143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:12.174143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:12.174143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:12.174143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:12.174143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:12.174143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:12.174143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:12.174143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:12.174143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:12.174143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:12.174941 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:12.174941 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:12.174941 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:12.174941 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:12.174941 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:12.174941 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:12.174941 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:12.174941 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:12.174941 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:12.174941 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:12.174941 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 23:30:12.174941 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:30:12.174941 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 82/164] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:30:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:12.175575 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab6116-0020-47ef-9dda-608d984d8ad3 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:12.302967 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab6116-0020-47ef-9dda-608d984d8ad3 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:12.306195 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:12.339673 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:12.346521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1126882e-ac70-4eaa-84f7-424fc8012377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:12.347177 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126882e-ac70-4eaa-84f7-424fc8012377 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:12.347455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126882e-ac70-4eaa-84f7-424fc8012377 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:12.347728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126882e-ac70-4eaa-84f7-424fc8012377 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:12.366446 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1126882e-ac70-4eaa-84f7-424fc8012377 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:12.371394 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-1126882e-ac70-4eaa-84f7-424fc8012377 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 23:30:12.371394 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:12.371394 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:12.371394 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:12.371394 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:12.371394 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:12.371394 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:12.371394 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:12.371394 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:12.371394 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:12.371394 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:12.371394 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:12.371394 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:12.371394 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:12.371394 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:12.371394 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:12.373200 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:12.373200 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:12.373200 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:12.373200 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:12.373200 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:12.373200 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:12.373200 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:12.373200 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:12.373200 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:12.373200 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:12.373200 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:12.373200 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:12.373200 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:12.373200 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:12.373200 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:12.373200 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:12.373984 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:12.373984 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:12.373984 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:12.373984 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:12.373984 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:12.373984 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:12.373984 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:12.373984 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:12.373984 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:12.373984 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:12.373984 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 23:30:12.373984 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:30:12.376675 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 83/165] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 17 23:30:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:12.377604 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1126882e-ac70-4eaa-84f7-424fc8012377 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:12.395656 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457f26a7-a18f-4c9e-a439-4a5511337c5e None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:12.463602 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93554ec-f5fc-4cc6-90dd-a541993236fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:12.465980 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93554ec-f5fc-4cc6-90dd-a541993236fe None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:12.467005 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93554ec-f5fc-4cc6-90dd-a541993236fe None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:12.476194 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93554ec-f5fc-4cc6-90dd-a541993236fe None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:12.477013 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93554ec-f5fc-4cc6-90dd-a541993236fe None admin] RBAC: Adding query filter params (name=member) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:12.477247 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93554ec-f5fc-4cc6-90dd-a541993236fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:12.478672 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93554ec-f5fc-4cc6-90dd-a541993236fe None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:12.495253 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 83/166] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 17 23:30:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:12.499995 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93554ec-f5fc-4cc6-90dd-a541993236fe None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:12.540071 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1126882e-ac70-4eaa-84f7-424fc8012377 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:12.583803 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e6d39a-bf42-4dc0-9845-32eab8743e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:12.584775 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6d39a-bf42-4dc0-9845-32eab8743e43 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:12.585257 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6d39a-bf42-4dc0-9845-32eab8743e43 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:12.585771 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6d39a-bf42-4dc0-9845-32eab8743e43 None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:12.586697 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e6d39a-bf42-4dc0-9845-32eab8743e43 None admin] RBAC: Adding query filter params (name=member) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:12.587299 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e6d39a-bf42-4dc0-9845-32eab8743e43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:12.589035 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e6d39a-bf42-4dc0-9845-32eab8743e43 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:12.604916 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 84/167] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 17 23:30:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:12.605671 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e6d39a-bf42-4dc0-9845-32eab8743e43 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:12.659338 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:12.661395 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93554ec-f5fc-4cc6-90dd-a541993236fe None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:12.705785 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be9824-88c4-46cd-9c73-408fad96be02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:12.706483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be9824-88c4-46cd-9c73-408fad96be02 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:12.706765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be9824-88c4-46cd-9c73-408fad96be02 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:12.707040 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be9824-88c4-46cd-9c73-408fad96be02 None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:12.707769 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15be9824-88c4-46cd-9c73-408fad96be02 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:12.708150 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15be9824-88c4-46cd-9c73-408fad96be02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:12.709559 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15be9824-88c4-46cd-9c73-408fad96be02 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:12.712157 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:12.721434 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 84/168] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:30:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:12.722318 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be9824-88c4-46cd-9c73-408fad96be02 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:12.826128 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e6d39a-bf42-4dc0-9845-32eab8743e43 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:12.828670 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be9824-88c4-46cd-9c73-408fad96be02 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:12.863602 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf78be0c-b810-4050-bf58-729f552b97ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:12.864186 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78be0c-b810-4050-bf58-729f552b97ad None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:12.864464 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78be0c-b810-4050-bf58-729f552b97ad None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:12.864793 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78be0c-b810-4050-bf58-729f552b97ad None admin] PATH_INFO: `/v3/users/45f6f1fc05fd4bf590127b863244444f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:12.865979 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf78be0c-b810-4050-bf58-729f552b97ad None admin] RBAC: Authorizing `identity:get_user(user_id=45f6f1fc05fd4bf590127b863244444f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:12.867631 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf78be0c-b810-4050-bf58-729f552b97ad None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:12.869567 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 85/169] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:30:12 2022] GET /identity/v3/users/45f6f1fc05fd4bf590127b863244444f => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:12.870340 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf78be0c-b810-4050-bf58-729f552b97ad None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:12.885989 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da591548-3e82-429f-90cc-06d876978bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:12.887035 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da591548-3e82-429f-90cc-06d876978bf7 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:12.887534 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da591548-3e82-429f-90cc-06d876978bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:12.888606 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da591548-3e82-429f-90cc-06d876978bf7 None admin] PATH_INFO: `/v3/users/45f6f1fc05fd4bf590127b863244444f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:12.890738 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da591548-3e82-429f-90cc-06d876978bf7 None admin] RBAC: Authorizing `identity:get_user(user_id=45f6f1fc05fd4bf590127b863244444f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:12.892457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da591548-3e82-429f-90cc-06d876978bf7 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:12.894373 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 85/170] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:30:12 2022] GET /identity/v3/users/45f6f1fc05fd4bf590127b863244444f => generated 275 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:12.895325 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da591548-3e82-429f-90cc-06d876978bf7 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:12.940320 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:12.974097 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf78be0c-b810-4050-bf58-729f552b97ad None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:13.009498 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cceaffe-678c-4238-8da3-918a54badb4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:13.010134 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cceaffe-678c-4238-8da3-918a54badb4f None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:13.010412 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cceaffe-678c-4238-8da3-918a54badb4f None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:13.010687 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cceaffe-678c-4238-8da3-918a54badb4f None admin] PATH_INFO: `/v3/users/45f6f1fc05fd4bf590127b863244444f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:13.012066 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cceaffe-678c-4238-8da3-918a54badb4f None admin] RBAC: Authorizing `identity:get_user(user_id=45f6f1fc05fd4bf590127b863244444f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:13.014462 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cceaffe-678c-4238-8da3-918a54badb4f None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:13.016390 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 86/171] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:30:12 2022] GET /identity/v3/users/45f6f1fc05fd4bf590127b863244444f => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:13.017319 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cceaffe-678c-4238-8da3-918a54badb4f None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:13.080977 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:13.096205 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da591548-3e82-429f-90cc-06d876978bf7 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:13.137392 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cceaffe-678c-4238-8da3-918a54badb4f None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:13.164812 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da3bdc4-9275-4e35-baad-8023c2d86d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:13.165873 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da3bdc4-9275-4e35-baad-8023c2d86d0d None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:13.166367 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da3bdc4-9275-4e35-baad-8023c2d86d0d None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:13.166842 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da3bdc4-9275-4e35-baad-8023c2d86d0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:13.176983 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b6783d-504a-4b92-b52c-cc871bc8c07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:13.177591 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6783d-504a-4b92-b52c-cc871bc8c07f None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:13.177869 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6783d-504a-4b92-b52c-cc871bc8c07f None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:13.178138 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6783d-504a-4b92-b52c-cc871bc8c07f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:13.184335 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:13.254437 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b6783d-504a-4b92-b52c-cc871bc8c07f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:13.255931 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b6783d-504a-4b92-b52c-cc871bc8c07f None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:13.307387 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da3bdc4-9275-4e35-baad-8023c2d86d0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:13.308905 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da3bdc4-9275-4e35-baad-8023c2d86d0d None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:13.384973 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 87/172] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:30:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:30:13.385843 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b6783d-504a-4b92-b52c-cc871bc8c07f None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:13.452425 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:13.476073 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:13.489495 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 86/173] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:30:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:30:13.490454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da3bdc4-9275-4e35-baad-8023c2d86d0d None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:13.572170 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b6783d-504a-4b92-b52c-cc871bc8c07f None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:13.619770 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42d5e8c-b7f0-472f-80ae-1d028a8f6380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:13.620413 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d5e8c-b7f0-472f-80ae-1d028a8f6380 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:13.620694 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d5e8c-b7f0-472f-80ae-1d028a8f6380 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:13.620959 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d5e8c-b7f0-472f-80ae-1d028a8f6380 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:13.679630 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da3bdc4-9275-4e35-baad-8023c2d86d0d None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:13.696391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42d5e8c-b7f0-472f-80ae-1d028a8f6380 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:13.697877 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42d5e8c-b7f0-472f-80ae-1d028a8f6380 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:13.734491 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:13.740646 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ece417-7fc7-4748-a7e8-ae5e0eb21656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:13.741526 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ece417-7fc7-4748-a7e8-ae5e0eb21656 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:13.742030 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ece417-7fc7-4748-a7e8-ae5e0eb21656 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:13.742504 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ece417-7fc7-4748-a7e8-ae5e0eb21656 None admin] PATH_INFO: `/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:13.743860 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ece417-7fc7-4748-a7e8-ae5e0eb21656 None admin] RBAC: Authorizing `identity:get_project(project_id=105a2b33e5ee4cf88b11c48aebf329d7)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:13.745530 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ece417-7fc7-4748-a7e8-ae5e0eb21656 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:13.747669 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 87/174] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:13 2022] GET /identity/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7 => generated 298 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:13.748674 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ece417-7fc7-4748-a7e8-ae5e0eb21656 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:13.808070 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 88/175] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:30:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:30:13.811780 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42d5e8c-b7f0-472f-80ae-1d028a8f6380 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:13.816982 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:13.856911 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ece417-7fc7-4748-a7e8-ae5e0eb21656 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:13.922891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42d5e8c-b7f0-472f-80ae-1d028a8f6380 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:13.944690 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cac6e0-2920-4ec4-9102-db25d8f50fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:13.945338 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cac6e0-2920-4ec4-9102-db25d8f50fcf None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:13.945616 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cac6e0-2920-4ec4-9102-db25d8f50fcf None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:13.945886 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cac6e0-2920-4ec4-9102-db25d8f50fcf None admin] PATH_INFO: `/v3/projects/f7d5d1f74f3448008365b515fc91db2c` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:13.947019 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cac6e0-2920-4ec4-9102-db25d8f50fcf None admin] RBAC: Authorizing `identity:get_project(project_id=f7d5d1f74f3448008365b515fc91db2c)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:13.948501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cac6e0-2920-4ec4-9102-db25d8f50fcf None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:13.950798 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 88/176] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:13 2022] GET /identity/v3/projects/f7d5d1f74f3448008365b515fc91db2c => generated 312 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:13.956857 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cac6e0-2920-4ec4-9102-db25d8f50fcf None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:13.967693 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ef7f6-b52d-4035-b6d0-08223d439dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:13.968438 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ef7f6-b52d-4035-b6d0-08223d439dcd None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:13.968720 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ef7f6-b52d-4035-b6d0-08223d439dcd None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:13.968994 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ef7f6-b52d-4035-b6d0-08223d439dcd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:13.969735 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ef7f6-b52d-4035-b6d0-08223d439dcd None admin] RBAC: Adding query filter params (role.id=1182317a465349478dde4e28cb7cf96f, scope.project.id=105a2b33e5ee4cf88b11c48aebf329d7, user.id=45f6f1fc05fd4bf590127b863244444f) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:13.970047 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ef7f6-b52d-4035-b6d0-08223d439dcd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:13.971502 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ef7f6-b52d-4035-b6d0-08223d439dcd None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:13.973388 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:13.979840 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 89/177] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:30:13 2022] GET /identity/v3/role_assignments?user.id=45f6f1fc05fd4bf590127b863244444f&scope.project.id=105a2b33e5ee4cf88b11c48aebf329d7&role.id=1182317a465349478dde4e28cb7cf96f => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:13.980507 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ef7f6-b52d-4035-b6d0-08223d439dcd None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:14.096116 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ef7f6-b52d-4035-b6d0-08223d439dcd None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:14.146844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ab3a8a-cbdb-4ccf-9cd9-66573a532102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:14.147468 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab3a8a-cbdb-4ccf-9cd9-66573a532102 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:14.147749 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab3a8a-cbdb-4ccf-9cd9-66573a532102 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:14.148054 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab3a8a-cbdb-4ccf-9cd9-66573a532102 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:14.148806 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ab3a8a-cbdb-4ccf-9cd9-66573a532102 None admin] RBAC: Adding query filter params (role.id=1182317a465349478dde4e28cb7cf96f, scope.project.id=f7d5d1f74f3448008365b515fc91db2c, user.id=45f6f1fc05fd4bf590127b863244444f) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:14.149125 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ab3a8a-cbdb-4ccf-9cd9-66573a532102 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:14.150719 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ab3a8a-cbdb-4ccf-9cd9-66573a532102 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:14.157685 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 90/178] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:30:13 2022] GET /identity/v3/role_assignments?user.id=45f6f1fc05fd4bf590127b863244444f&scope.project.id=f7d5d1f74f3448008365b515fc91db2c&role.id=1182317a465349478dde4e28cb7cf96f => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:14.158662 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cac6e0-2920-4ec4-9102-db25d8f50fcf None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:14.158930 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1d387-f189-4cf5-964f-3a1c5f5346b1 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:14.159224 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1d387-f189-4cf5-964f-3a1c5f5346b1 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:14.159500 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1d387-f189-4cf5-964f-3a1c5f5346b1 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:14.165041 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 91/179] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:14.176024 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b8ddf-f178-45b5-9126-4d68fec7ac09 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:14.176370 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b8ddf-f178-45b5-9126-4d68fec7ac09 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:14.176576 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b8ddf-f178-45b5-9126-4d68fec7ac09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:14.209310 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-661b8ddf-f178-45b5-9126-4d68fec7ac09 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:14.210548 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-661b8ddf-f178-45b5-9126-4d68fec7ac09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:14.234271 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:14.268801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f935aa9f-5046-4529-b8af-916d1c493120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:14.269658 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935aa9f-5046-4529-b8af-916d1c493120 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:14.269766 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935aa9f-5046-4529-b8af-916d1c493120 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:14.270002 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935aa9f-5046-4529-b8af-916d1c493120 None admin] PATH_INFO: `/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:14.272131 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f935aa9f-5046-4529-b8af-916d1c493120 None admin] RBAC: Authorizing `identity:get_project(project_id=105a2b33e5ee4cf88b11c48aebf329d7)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:14.273626 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f935aa9f-5046-4529-b8af-916d1c493120 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:14.275663 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 89/180] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:13 2022] GET /identity/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7 => generated 298 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:14.282755 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f935aa9f-5046-4529-b8af-916d1c493120 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:14.396260 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:14.444922 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 92/181] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:14.492064 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f935aa9f-5046-4529-b8af-916d1c493120 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:14.543435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c6239-163e-4011-ac5d-70d962c07c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:14.544611 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c6239-163e-4011-ac5d-70d962c07c0d None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:14.544891 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c6239-163e-4011-ac5d-70d962c07c0d None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:14.545163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c6239-163e-4011-ac5d-70d962c07c0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:14.545898 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37c6239-163e-4011-ac5d-70d962c07c0d None admin] RBAC: Adding query filter params (role.id=3428ff8e92274de6817b7e8ee9a56d13, scope.project.id=105a2b33e5ee4cf88b11c48aebf329d7, user.id=45f6f1fc05fd4bf590127b863244444f) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:14.546198 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37c6239-163e-4011-ac5d-70d962c07c0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:14.547665 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37c6239-163e-4011-ac5d-70d962c07c0d None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:14.554207 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 90/182] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:30:14 2022] GET /identity/v3/role_assignments?user.id=45f6f1fc05fd4bf590127b863244444f&scope.project.id=105a2b33e5ee4cf88b11c48aebf329d7&role.id=3428ff8e92274de6817b7e8ee9a56d13 => generated 608 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:14.853395 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b8ddf-f178-45b5-9126-4d68fec7ac09 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:14.904749 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801bd53-312b-45bb-8867-f3c6aa217122 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:14.905267 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801bd53-312b-45bb-8867-f3c6aa217122 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:14.905267 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801bd53-312b-45bb-8867-f3c6aa217122 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:14.906638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 91/183] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:14.912397 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c949a1e-902f-4c1b-a6ff-1e87bc1ad7ca None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:14.912681 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c949a1e-902f-4c1b-a6ff-1e87bc1ad7ca None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:14.912960 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c949a1e-902f-4c1b-a6ff-1e87bc1ad7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:14.948693 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-6c949a1e-902f-4c1b-a6ff-1e87bc1ad7ca None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:14.949937 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-6c949a1e-902f-4c1b-a6ff-1e87bc1ad7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:15.022893 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:15.031763 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b8ddf-f178-45b5-9126-4d68fec7ac09 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:15.044309 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 92/184] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 23:30:15.046568 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95158a75-9364-411f-8cac-b24e494b7d33 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:15.046568 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95158a75-9364-411f-8cac-b24e494b7d33 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:15.046568 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95158a75-9364-411f-8cac-b24e494b7d33 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:15.048096 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 93/185] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:15.053973 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197fe97-0fc5-4470-a954-2240c6e90594 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:15.054263 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197fe97-0fc5-4470-a954-2240c6e90594 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:15.054533 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197fe97-0fc5-4470-a954-2240c6e90594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:15.078968 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-0197fe97-0fc5-4470-a954-2240c6e90594 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:15.080192 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-0197fe97-0fc5-4470-a954-2240c6e90594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:15.125703 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f5ea2f-09d9-47a3-840e-dbc99a4162ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:15.126318 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f5ea2f-09d9-47a3-840e-dbc99a4162ca None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:15.126655 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f5ea2f-09d9-47a3-840e-dbc99a4162ca None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:15.126925 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f5ea2f-09d9-47a3-840e-dbc99a4162ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:15.151262 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f5ea2f-09d9-47a3-840e-dbc99a4162ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:15.157735 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-57f5ea2f-09d9-47a3-840e-dbc99a4162ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:30:15.157735 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:15.157735 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:15.157735 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:15.157735 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:15.157735 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:15.157735 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:15.157735 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:15.157735 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:15.157735 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:15.157735 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:15.157735 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:15.157735 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:15.157735 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:15.157735 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:15.157735 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:15.158801 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:15.158801 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:15.158801 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:15.158801 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:15.158801 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:15.158801 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:15.158801 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:15.158801 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:15.158801 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:15.158801 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:15.158801 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:15.158801 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:15.158801 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:15.158801 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:15.158801 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:15.158801 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:15.159891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:15.159891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:15.159891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:15.159891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:15.159891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:15.159891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:15.159891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:15.159891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:15.159891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:15.159891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:15.159891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:30:15.159891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:30:15.159891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 93/186] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 17 23:30:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:15.163989 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f5ea2f-09d9-47a3-840e-dbc99a4162ca None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:15.189178 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 94/187] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:15.235019 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:15.294755 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f5ea2f-09d9-47a3-840e-dbc99a4162ca None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:15.338459 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67682bb1-bab7-45f5-8457-287dbbd9704a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:15.339333 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67682bb1-bab7-45f5-8457-287dbbd9704a None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:15.339759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67682bb1-bab7-45f5-8457-287dbbd9704a None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:15.340277 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67682bb1-bab7-45f5-8457-287dbbd9704a None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:15.341133 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67682bb1-bab7-45f5-8457-287dbbd9704a None admin] RBAC: Adding query filter params (name=admin) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:15.341571 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67682bb1-bab7-45f5-8457-287dbbd9704a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:15.343106 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67682bb1-bab7-45f5-8457-287dbbd9704a None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:15.352219 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 94/188] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:30:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:15.358164 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0197fe97-0fc5-4470-a954-2240c6e90594 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:15.365642 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67682bb1-bab7-45f5-8457-287dbbd9704a None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:15.486620 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67682bb1-bab7-45f5-8457-287dbbd9704a None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:15.489901 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0197fe97-0fc5-4470-a954-2240c6e90594 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:15.502095 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:15.516348 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:15.524599 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b97c1aa-67c5-4272-8448-26009a418af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:15.525235 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97c1aa-67c5-4272-8448-26009a418af6 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:15.525516 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97c1aa-67c5-4272-8448-26009a418af6 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:15.525784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97c1aa-67c5-4272-8448-26009a418af6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:15.528150 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1edbb-6496-4bbb-bbbd-3977058cf056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:15.528784 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1edbb-6496-4bbb-bbbd-3977058cf056 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:15.529066 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1edbb-6496-4bbb-bbbd-3977058cf056 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:15.529342 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1edbb-6496-4bbb-bbbd-3977058cf056 None admin] PATH_INFO: `/v3/users/6530156892a74973822b17cdc23f6faa` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:15.537342 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b97c1aa-67c5-4272-8448-26009a418af6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:15.542589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-6b97c1aa-67c5-4272-8448-26009a418af6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 23:30:15.542589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:15.542589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:15.542589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:15.542589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:15.542589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:15.542589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:15.542589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:15.542589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:15.542589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:15.542589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:15.542589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:15.542589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:15.542589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:15.542589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:15.542589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:15.543556 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:15.543556 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:15.543556 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:15.543556 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:15.543556 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:15.543556 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:15.543556 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:15.543556 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:15.543556 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:15.543556 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:15.543556 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:15.543556 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:15.543556 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:15.543556 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:15.543556 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:15.543556 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:15.544929 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:15.544929 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:15.544929 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:15.544929 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:15.544929 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:15.544929 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:15.544929 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:15.544929 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:15.544929 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:15.544929 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:15.544929 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 23:30:15.544929 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:30:15.544929 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 95/189] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 17 23:30:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:30:15.545589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f2e4a-93ec-4ac4-94e2-49df34f15fe6 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:15.545952 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f2e4a-93ec-4ac4-94e2-49df34f15fe6 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:15.546126 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f2e4a-93ec-4ac4-94e2-49df34f15fe6 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:15.548686 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 96/190] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:15.548686 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f2e4a-93ec-4ac4-94e2-49df34f15fe6 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:15.549767 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1edbb-6496-4bbb-bbbd-3977058cf056 None admin] RBAC: Authorizing `identity:get_user(user_id=6530156892a74973822b17cdc23f6faa)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:15.551327 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1edbb-6496-4bbb-bbbd-3977058cf056 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:15.556838 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 95/191] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:30:15 2022] GET /identity/v3/users/6530156892a74973822b17cdc23f6faa => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:15.556838 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1edbb-6496-4bbb-bbbd-3977058cf056 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:15.661473 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f2e4a-93ec-4ac4-94e2-49df34f15fe6 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:15.668159 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1edbb-6496-4bbb-bbbd-3977058cf056 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:15.697933 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2102267e-7df3-4a67-8d2d-91487b8124ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:15.698564 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102267e-7df3-4a67-8d2d-91487b8124ff None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:15.698843 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102267e-7df3-4a67-8d2d-91487b8124ff None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:15.699114 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102267e-7df3-4a67-8d2d-91487b8124ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:15.700257 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2102267e-7df3-4a67-8d2d-91487b8124ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:15.701704 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2102267e-7df3-4a67-8d2d-91487b8124ff None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:15.707408 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 97/192] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:30:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:15.707553 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaceacd3-eef3-4755-90dc-619da66aa26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:15.707666 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47925bde-55b4-4b8b-a986-ca255b1d0164 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:15.707666 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47925bde-55b4-4b8b-a986-ca255b1d0164 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:15.707666 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47925bde-55b4-4b8b-a986-ca255b1d0164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:15.707850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaceacd3-eef3-4755-90dc-619da66aa26f None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:15.707850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaceacd3-eef3-4755-90dc-619da66aa26f None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:15.707850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaceacd3-eef3-4755-90dc-619da66aa26f None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:15.707850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaceacd3-eef3-4755-90dc-619da66aa26f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:15.707850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaceacd3-eef3-4755-90dc-619da66aa26f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:15.707850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaceacd3-eef3-4755-90dc-619da66aa26f None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:15.717178 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 96/193] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:30:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:15.718322 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaceacd3-eef3-4755-90dc-619da66aa26f None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:15.724136 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-47925bde-55b4-4b8b-a986-ca255b1d0164 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:15.725432 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-47925bde-55b4-4b8b-a986-ca255b1d0164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:15.762573 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:15.764482 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:15.796042 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 98/194] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:15.804866 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47925bde-55b4-4b8b-a986-ca255b1d0164 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:15.826972 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaceacd3-eef3-4755-90dc-619da66aa26f None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:15.871119 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a9b1e-98e0-4dab-887a-82a2bcb910e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:15.871738 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a9b1e-98e0-4dab-887a-82a2bcb910e4 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:15.872046 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a9b1e-98e0-4dab-887a-82a2bcb910e4 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:15.872320 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a9b1e-98e0-4dab-887a-82a2bcb910e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:15.922288 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47925bde-55b4-4b8b-a986-ca255b1d0164 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:15.947749 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070a9b1e-98e0-4dab-887a-82a2bcb910e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:15.949261 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070a9b1e-98e0-4dab-887a-82a2bcb910e4 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:15.961404 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a82f174-d231-4221-89f6-80a1d9ced0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:15.961966 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a82f174-d231-4221-89f6-80a1d9ced0ec None admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:15.962249 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a82f174-d231-4221-89f6-80a1d9ced0ec None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:15.962821 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a82f174-d231-4221-89f6-80a1d9ced0ec None admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:15.963478 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a82f174-d231-4221-89f6-80a1d9ced0ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:15.964977 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a82f174-d231-4221-89f6-80a1d9ced0ec None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:15.987693 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 99/195] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Aug 17 23:30:15 2022] POST /identity/v3/users => generated 297 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:30:15.997262 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a82f174-d231-4221-89f6-80a1d9ced0ec None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:15.997764 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:16.033206 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:16.085519 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 97/196] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:30:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:30:16.085519 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a9b1e-98e0-4dab-887a-82a2bcb910e4 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:16.133909 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a82f174-d231-4221-89f6-80a1d9ced0ec None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:16.177362 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8fab81-806b-4ee5-a622-c080dacc8e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:16.178161 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8fab81-806b-4ee5-a622-c080dacc8e96 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:16.178445 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8fab81-806b-4ee5-a622-c080dacc8e96 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:16.178794 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8fab81-806b-4ee5-a622-c080dacc8e96 None admin] PATH_INFO: `/v3/users/6530156892a74973822b17cdc23f6faa` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:16.180115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8fab81-806b-4ee5-a622-c080dacc8e96 None admin] RBAC: Authorizing `identity:get_user(user_id=6530156892a74973822b17cdc23f6faa)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:16.181581 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8fab81-806b-4ee5-a622-c080dacc8e96 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:16.183375 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 100/197] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:30:15 2022] GET /identity/v3/users/6530156892a74973822b17cdc23f6faa => generated 283 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:16.188688 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8fab81-806b-4ee5-a622-c080dacc8e96 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:16.198285 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a9b1e-98e0-4dab-887a-82a2bcb910e4 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:16.247645 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163cf659-14ae-468e-b538-30234dcf2754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:16.248644 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163cf659-14ae-468e-b538-30234dcf2754 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:16.248840 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163cf659-14ae-468e-b538-30234dcf2754 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:16.249111 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163cf659-14ae-468e-b538-30234dcf2754 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:16.264125 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163cf659-14ae-468e-b538-30234dcf2754 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:16.270221 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-163cf659-14ae-468e-b538-30234dcf2754 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 23:30:16.270221 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:16.270221 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:16.270221 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:16.270221 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:16.270221 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:16.270221 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:16.270221 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:16.270221 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:16.270221 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:16.270221 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:16.270221 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:16.270221 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:16.270221 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:16.270221 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:16.270221 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:16.271092 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:16.271092 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:16.271092 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:16.271092 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:16.271092 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:16.271092 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:16.271092 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:16.271092 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:16.271092 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:16.271092 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:16.271092 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:16.271092 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:16.271092 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:16.271092 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:16.271092 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:16.271092 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:16.271864 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:16.271864 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:16.271864 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:16.271864 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:16.271864 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:16.271864 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:16.271864 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:16.271864 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:16.271864 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:16.271864 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:16.271864 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 23:30:16.271864 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:30:16.272851 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 98/198] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:30:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:16.272851 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163cf659-14ae-468e-b538-30234dcf2754 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:16.285675 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:16.318980 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:16.326158 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8fab81-806b-4ee5-a622-c080dacc8e96 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:16.369101 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50e70c-1d8e-48d6-a2ec-2446b300254d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:16.369741 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50e70c-1d8e-48d6-a2ec-2446b300254d None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:16.370022 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50e70c-1d8e-48d6-a2ec-2446b300254d None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:16.371038 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50e70c-1d8e-48d6-a2ec-2446b300254d None admin] PATH_INFO: `/v3/projects/e959a87f43754d66b9bd795c08d2d827` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:16.371644 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd50e70c-1d8e-48d6-a2ec-2446b300254d None admin] RBAC: Authorizing `identity:get_project(project_id=e959a87f43754d66b9bd795c08d2d827)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:16.373686 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd50e70c-1d8e-48d6-a2ec-2446b300254d None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:16.375503 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 101/199] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:16 2022] GET /identity/v3/projects/e959a87f43754d66b9bd795c08d2d827 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:16.377554 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50e70c-1d8e-48d6-a2ec-2446b300254d None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:16.421680 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163cf659-14ae-468e-b538-30234dcf2754 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:16.470735 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c28aa-b4fa-4676-92f7-93e8e1949004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:16.471580 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c28aa-b4fa-4676-92f7-93e8e1949004 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:16.472242 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c28aa-b4fa-4676-92f7-93e8e1949004 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:16.472660 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c28aa-b4fa-4676-92f7-93e8e1949004 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:16.503806 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50e70c-1d8e-48d6-a2ec-2446b300254d None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:16.544396 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162a00a-5cf8-4542-8750-655c966ecdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:16.545149 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162a00a-5cf8-4542-8750-655c966ecdb3 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:16.545411 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162a00a-5cf8-4542-8750-655c966ecdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:16.545686 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162a00a-5cf8-4542-8750-655c966ecdb3 None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:16.546368 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5162a00a-5cf8-4542-8750-655c966ecdb3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:16.546747 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5162a00a-5cf8-4542-8750-655c966ecdb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:16.548264 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5162a00a-5cf8-4542-8750-655c966ecdb3 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:16.557652 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 102/200] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 17 23:30:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:16.558305 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162a00a-5cf8-4542-8750-655c966ecdb3 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:16.565305 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c28aa-b4fa-4676-92f7-93e8e1949004 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:16.568357 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c28aa-b4fa-4676-92f7-93e8e1949004 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:16.570486 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:16.601241 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:16.677660 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162a00a-5cf8-4542-8750-655c966ecdb3 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:16.698341 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 99/201] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:30:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:30:16.699418 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c28aa-b4fa-4676-92f7-93e8e1949004 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:16.716585 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e9af9-e95f-4855-82c6-1362bb0a6277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:16.717195 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e9af9-e95f-4855-82c6-1362bb0a6277 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:16.717849 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e9af9-e95f-4855-82c6-1362bb0a6277 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:16.718120 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e9af9-e95f-4855-82c6-1362bb0a6277 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:16.719121 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47e9af9-e95f-4855-82c6-1362bb0a6277 None admin] RBAC: Adding query filter params (role.id=33bd60d3b0e94dcbbfe97e87f91a8d78, scope.project.id=e959a87f43754d66b9bd795c08d2d827, user.id=6530156892a74973822b17cdc23f6faa) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:16.719427 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47e9af9-e95f-4855-82c6-1362bb0a6277 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:16.721624 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47e9af9-e95f-4855-82c6-1362bb0a6277 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:16.728709 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 103/202] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:30:16 2022] GET /identity/v3/role_assignments?user.id=6530156892a74973822b17cdc23f6faa&scope.project.id=e959a87f43754d66b9bd795c08d2d827&role.id=33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:16.729499 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e9af9-e95f-4855-82c6-1362bb0a6277 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:16.844978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:16.853137 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e9af9-e95f-4855-82c6-1362bb0a6277 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:16.894943 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af987e0-9ab0-4023-967b-d157ce7fd0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:16.895592 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af987e0-9ab0-4023-967b-d157ce7fd0ce None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:16.895876 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af987e0-9ab0-4023-967b-d157ce7fd0ce None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:16.896200 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af987e0-9ab0-4023-967b-d157ce7fd0ce None admin] PATH_INFO: `/v3/projects/e959a87f43754d66b9bd795c08d2d827` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:16.897536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af987e0-9ab0-4023-967b-d157ce7fd0ce None admin] RBAC: Authorizing `identity:get_project(project_id=e959a87f43754d66b9bd795c08d2d827)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:16.899144 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af987e0-9ab0-4023-967b-d157ce7fd0ce None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:16.901340 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 104/203] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:16 2022] GET /identity/v3/projects/e959a87f43754d66b9bd795c08d2d827 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:16.909638 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af987e0-9ab0-4023-967b-d157ce7fd0ce None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:16.947128 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c28aa-b4fa-4676-92f7-93e8e1949004 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:16.948908 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:17.008978 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0514d72f-e66d-4d53-b158-51b258470c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:17.009613 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514d72f-e66d-4d53-b158-51b258470c61 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:17.009889 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514d72f-e66d-4d53-b158-51b258470c61 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:17.010160 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514d72f-e66d-4d53-b158-51b258470c61 None admin] PATH_INFO: `/v3/users/75bac2f681594654bfb749088ae12b19` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:17.011852 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0514d72f-e66d-4d53-b158-51b258470c61 None admin] RBAC: Authorizing `identity:get_user(user_id=75bac2f681594654bfb749088ae12b19)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:17.013401 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0514d72f-e66d-4d53-b158-51b258470c61 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:17.014973 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 100/204] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:30:16 2022] GET /identity/v3/users/75bac2f681594654bfb749088ae12b19 => generated 289 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:17.022418 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0514d72f-e66d-4d53-b158-51b258470c61 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:17.091908 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af987e0-9ab0-4023-967b-d157ce7fd0ce None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:17.153002 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a260df13-86d2-4f17-8cc6-858fcc4fa2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:17.155249 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a260df13-86d2-4f17-8cc6-858fcc4fa2d3 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:17.158563 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a260df13-86d2-4f17-8cc6-858fcc4fa2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:17.159138 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a260df13-86d2-4f17-8cc6-858fcc4fa2d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:17.160104 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a260df13-86d2-4f17-8cc6-858fcc4fa2d3 None admin] RBAC: Adding query filter params (role.id=3428ff8e92274de6817b7e8ee9a56d13, scope.project.id=e959a87f43754d66b9bd795c08d2d827, user.id=6530156892a74973822b17cdc23f6faa) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:17.160617 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a260df13-86d2-4f17-8cc6-858fcc4fa2d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:17.162287 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a260df13-86d2-4f17-8cc6-858fcc4fa2d3 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:17.174592 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 105/205] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:30:16 2022] GET /identity/v3/role_assignments?user.id=6530156892a74973822b17cdc23f6faa&scope.project.id=e959a87f43754d66b9bd795c08d2d827&role.id=3428ff8e92274de6817b7e8ee9a56d13 => generated 259 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:30:17.180968 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0514d72f-e66d-4d53-b158-51b258470c61 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:17.245008 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbde328-d14f-4af2-b5af-e6c65e24b40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:17.245979 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbde328-d14f-4af2-b5af-e6c65e24b40e None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:17.246485 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbde328-d14f-4af2-b5af-e6c65e24b40e None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:17.247540 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbde328-d14f-4af2-b5af-e6c65e24b40e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:17.331823 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:17.385068 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbde328-d14f-4af2-b5af-e6c65e24b40e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:17.387193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbde328-d14f-4af2-b5af-e6c65e24b40e None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:17.506757 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 101/206] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:30:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:30:17.512323 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a260df13-86d2-4f17-8cc6-858fcc4fa2d3 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:17.561166 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:17.686042 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a260df13-86d2-4f17-8cc6-858fcc4fa2d3 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:17.748684 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab050e02-35f7-48a3-a45b-71b1b6a4556d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:17.749334 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab050e02-35f7-48a3-a45b-71b1b6a4556d None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:17.749622 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab050e02-35f7-48a3-a45b-71b1b6a4556d None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:17.749899 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab050e02-35f7-48a3-a45b-71b1b6a4556d None admin] PATH_INFO: `/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:17.751078 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab050e02-35f7-48a3-a45b-71b1b6a4556d None admin] RBAC: Authorizing `identity:get_project(project_id=105a2b33e5ee4cf88b11c48aebf329d7)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:17.752570 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab050e02-35f7-48a3-a45b-71b1b6a4556d None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:17.754539 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 106/207] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:17 2022] GET /identity/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7 => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:17.761048 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbde328-d14f-4af2-b5af-e6c65e24b40e None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:17.921826 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:17.986847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbde328-d14f-4af2-b5af-e6c65e24b40e None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:18.048836 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763100f0-9f6b-4efa-9948-a0a5ea33cbab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:18.049834 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763100f0-9f6b-4efa-9948-a0a5ea33cbab None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:18.050377 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763100f0-9f6b-4efa-9948-a0a5ea33cbab None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:18.050845 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763100f0-9f6b-4efa-9948-a0a5ea33cbab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:18.051783 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763100f0-9f6b-4efa-9948-a0a5ea33cbab None admin] RBAC: Adding query filter params (role.id=df3442912b3d43f0a36ebb936b3cb102, scope.project.id=105a2b33e5ee4cf88b11c48aebf329d7, user.id=75bac2f681594654bfb749088ae12b19) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:18.052304 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763100f0-9f6b-4efa-9948-a0a5ea33cbab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:18.053949 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763100f0-9f6b-4efa-9948-a0a5ea33cbab None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:18.071420 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 102/208] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:30:17 2022] GET /identity/v3/role_assignments?user.id=75bac2f681594654bfb749088ae12b19&scope.project.id=105a2b33e5ee4cf88b11c48aebf329d7&role.id=df3442912b3d43f0a36ebb936b3cb102 => generated 608 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:18.327763 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549ea29-583a-411a-bd77-5b92d568990b None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:18.328288 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549ea29-583a-411a-bd77-5b92d568990b None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:18.328288 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549ea29-583a-411a-bd77-5b92d568990b None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:18.329910 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 107/209] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:18.335515 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b38bb1-b495-4580-980b-e8ab977f72fe None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:18.335806 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b38bb1-b495-4580-980b-e8ab977f72fe None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:18.336111 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b38bb1-b495-4580-980b-e8ab977f72fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:18.363947 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-94b38bb1-b495-4580-980b-e8ab977f72fe None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:18.365176 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-94b38bb1-b495-4580-980b-e8ab977f72fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:18.381612 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e4134-e47d-45b4-a2ad-96105ab7c35e None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:18.381895 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e4134-e47d-45b4-a2ad-96105ab7c35e None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:18.382174 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e4134-e47d-45b4-a2ad-96105ab7c35e None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:18.383519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 108/210] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:18.389199 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240738a-ffbf-4f51-a619-c7a53f5495f0 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:18.389593 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240738a-ffbf-4f51-a619-c7a53f5495f0 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:18.389943 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240738a-ffbf-4f51-a619-c7a53f5495f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:18.413759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-8240738a-ffbf-4f51-a619-c7a53f5495f0 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:18.415079 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-8240738a-ffbf-4f51-a619-c7a53f5495f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:18.484863 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 103/211] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 23:30:18.519418 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 109/212] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 23:30:18.770547 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b38bb1-b495-4580-980b-e8ab977f72fe None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:18.806853 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8240738a-ffbf-4f51-a619-c7a53f5495f0 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:18.835143 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:18.850615 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:18.898168 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b38bb1-b495-4580-980b-e8ab977f72fe None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:18.926507 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8240738a-ffbf-4f51-a619-c7a53f5495f0 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:18.942645 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109147f6-977d-4b70-8b89-280ddfbfe41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:18.943820 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109147f6-977d-4b70-8b89-280ddfbfe41e None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:18.944300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109147f6-977d-4b70-8b89-280ddfbfe41e None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:18.944909 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109147f6-977d-4b70-8b89-280ddfbfe41e None admin] PATH_INFO: `/v3/domains/default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:18.946424 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109147f6-977d-4b70-8b89-280ddfbfe41e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:18.948089 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109147f6-977d-4b70-8b89-280ddfbfe41e None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:18.955244 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 104/213] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:30:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:18.955918 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109147f6-977d-4b70-8b89-280ddfbfe41e None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:18.970688 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6acef-c815-4099-bdec-f593226a6840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:18.971740 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6acef-c815-4099-bdec-f593226a6840 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:18.972392 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6acef-c815-4099-bdec-f593226a6840 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:18.973227 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6acef-c815-4099-bdec-f593226a6840 None admin] PATH_INFO: `/v3/roles/member` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:18.989288 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a6acef-c815-4099-bdec-f593226a6840 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:18.994370 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-a9a6acef-c815-4099-bdec-f593226a6840 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 23:30:18.994370 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:18.994370 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:18.994370 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:18.994370 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:18.994370 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:18.994370 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:18.994370 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:18.994370 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:18.994370 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:18.994370 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:18.994370 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:18.994370 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:18.994370 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:18.994370 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:18.994370 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:18.995237 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:18.995237 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:18.995237 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:18.995237 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:18.995237 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:18.995237 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:18.995237 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:18.995237 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:18.995237 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:18.995237 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:18.995237 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:18.995237 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:18.995237 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:18.995237 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:18.995237 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:18.995237 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:18.996058 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:18.996058 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:18.996058 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:18.996058 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:18.996058 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:18.996058 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:18.996058 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:18.996058 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:18.996058 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:18.996058 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:18.996058 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 23:30:18.996058 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:30:19.000241 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 110/214] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:30:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:19.001665 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6acef-c815-4099-bdec-f593226a6840 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:19.076971 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109147f6-977d-4b70-8b89-280ddfbfe41e None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:19.091837 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:19.111432 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:19.115515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6acef-c815-4099-bdec-f593226a6840 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:19.118487 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059ac9ef-604b-4496-bc0c-d7989c860b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:19.119212 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ac9ef-604b-4496-bc0c-d7989c860b4c None admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:19.119677 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ac9ef-604b-4496-bc0c-d7989c860b4c None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:19.120036 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ac9ef-604b-4496-bc0c-d7989c860b4c None admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:19.120763 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059ac9ef-604b-4496-bc0c-d7989c860b4c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:19.122394 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059ac9ef-604b-4496-bc0c-d7989c860b4c None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:19.152164 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ae31a-9e3d-4f2f-95a5-394943aedbb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:19.154006 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ae31a-9e3d-4f2f-95a5-394943aedbb9 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:19.154494 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ae31a-9e3d-4f2f-95a5-394943aedbb9 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:19.154966 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ae31a-9e3d-4f2f-95a5-394943aedbb9 None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:19.155869 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83ae31a-9e3d-4f2f-95a5-394943aedbb9 None admin] RBAC: Adding query filter params (name=member) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:19.156194 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 105/215] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Aug 17 23:30:18 2022] POST /identity/v3/users => generated 297 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:30:19.156522 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83ae31a-9e3d-4f2f-95a5-394943aedbb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:19.157634 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1db2e9-1faa-48cf-84a9-c2e848424227 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:19.157916 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1db2e9-1faa-48cf-84a9-c2e848424227 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:19.158188 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1db2e9-1faa-48cf-84a9-c2e848424227 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:19.158602 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83ae31a-9e3d-4f2f-95a5-394943aedbb9 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:19.159511 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 106/216] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:19.170104 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c14cd-6104-416e-9ddd-cf288a467047 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:19.170420 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c14cd-6104-416e-9ddd-cf288a467047 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:19.170503 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c14cd-6104-416e-9ddd-cf288a467047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:19.174654 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 111/217] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 17 23:30:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:19.181004 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ae31a-9e3d-4f2f-95a5-394943aedbb9 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:19.198059 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-df5c14cd-6104-416e-9ddd-cf288a467047 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:19.201264 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-df5c14cd-6104-416e-9ddd-cf288a467047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:19.290103 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 107/218] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:19.310622 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ae31a-9e3d-4f2f-95a5-394943aedbb9 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:19.354988 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad0287-20dc-411e-be9a-ff04b5c888db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:19.355593 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad0287-20dc-411e-be9a-ff04b5c888db None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:19.355916 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad0287-20dc-411e-be9a-ff04b5c888db None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:19.356221 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad0287-20dc-411e-be9a-ff04b5c888db None admin] PATH_INFO: `/v3/users/6dfd2578b49e454fa81bb5d1b9ae3b72` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:19.366266 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd2bd4-a114-45d9-a704-415236430f11 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:19.366548 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd2bd4-a114-45d9-a704-415236430f11 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:19.366820 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd2bd4-a114-45d9-a704-415236430f11 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:19.368105 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:19.370624 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 108/219] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:19.371784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fad0287-20dc-411e-be9a-ff04b5c888db None admin] RBAC: Authorizing `identity:get_user(user_id=6dfd2578b49e454fa81bb5d1b9ae3b72)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:19.374691 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fad0287-20dc-411e-be9a-ff04b5c888db None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:19.377124 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 112/220] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:30:19 2022] GET /identity/v3/users/6dfd2578b49e454fa81bb5d1b9ae3b72 => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:19.378111 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070e426-57fb-4f6c-84d9-03d4885621cb None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:19.378596 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070e426-57fb-4f6c-84d9-03d4885621cb None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:19.378754 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070e426-57fb-4f6c-84d9-03d4885621cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:19.381926 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad0287-20dc-411e-be9a-ff04b5c888db None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:19.406633 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-4070e426-57fb-4f6c-84d9-03d4885621cb None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:19.408157 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-4070e426-57fb-4f6c-84d9-03d4885621cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:19.534148 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 109/221] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:19.534901 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad0287-20dc-411e-be9a-ff04b5c888db None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:19.593479 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd13bca-8750-46a3-a9c6-a8bcb2e600a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:19.593479 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd13bca-8750-46a3-a9c6-a8bcb2e600a4 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:19.593757 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd13bca-8750-46a3-a9c6-a8bcb2e600a4 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:19.594015 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd13bca-8750-46a3-a9c6-a8bcb2e600a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:19.595537 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070e426-57fb-4f6c-84d9-03d4885621cb None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:19.684993 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:19.689689 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd13bca-8750-46a3-a9c6-a8bcb2e600a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:19.690972 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd13bca-8750-46a3-a9c6-a8bcb2e600a4 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:19.691600 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:19.805233 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070e426-57fb-4f6c-84d9-03d4885621cb None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:19.872086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155df3ee-0d3e-418f-9646-e03459dccc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:19.872735 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155df3ee-0d3e-418f-9646-e03459dccc30 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:19.873014 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155df3ee-0d3e-418f-9646-e03459dccc30 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:19.873287 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155df3ee-0d3e-418f-9646-e03459dccc30 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:19.892353 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155df3ee-0d3e-418f-9646-e03459dccc30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:19.901947 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-155df3ee-0d3e-418f-9646-e03459dccc30 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:30:19.901947 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:19.901947 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:19.901947 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:19.901947 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:19.901947 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:19.901947 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:19.901947 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:19.901947 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:19.901947 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:19.901947 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:19.901947 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:19.901947 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:19.901947 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:19.901947 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:19.901947 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:19.903122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:19.903122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:19.903122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:19.903122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:19.903122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:19.903122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:19.903122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:19.903122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:19.903122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:19.903122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:19.903122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:19.903122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:19.903122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:19.903122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:19.903122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:19.903122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:19.904484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:19.904484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:19.904484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:19.904484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:19.904484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:19.904484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:19.904484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:19.904484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:19.904484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:19.904484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:19.904484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:30:19.904484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:30:19.904484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 110/222] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 17 23:30:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:30:19.905129 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155df3ee-0d3e-418f-9646-e03459dccc30 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:19.976254 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 113/223] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:30:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:30:19.977639 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd13bca-8750-46a3-a9c6-a8bcb2e600a4 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:20.097983 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155df3ee-0d3e-418f-9646-e03459dccc30 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:20.120119 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:20.179687 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:20.184457 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd13bca-8750-46a3-a9c6-a8bcb2e600a4 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:20.221288 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5f1f8-086b-4b35-a9fd-f5b6a486c1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:20.221937 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5f1f8-086b-4b35-a9fd-f5b6a486c1eb None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:20.222212 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5f1f8-086b-4b35-a9fd-f5b6a486c1eb None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:20.222481 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5f1f8-086b-4b35-a9fd-f5b6a486c1eb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:20.244050 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f5f1f8-086b-4b35-a9fd-f5b6a486c1eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:20.244183 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef336c05-170b-411d-9ed0-93daa0429f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:20.244183 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef336c05-170b-411d-9ed0-93daa0429f56 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:20.244183 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef336c05-170b-411d-9ed0-93daa0429f56 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:20.244183 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef336c05-170b-411d-9ed0-93daa0429f56 None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:20.244454 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef336c05-170b-411d-9ed0-93daa0429f56 None admin] RBAC: Adding query filter params (name=admin) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:20.244701 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef336c05-170b-411d-9ed0-93daa0429f56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:20.250041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-a5f5f1f8-086b-4b35-a9fd-f5b6a486c1eb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 23:30:20.250041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:20.250041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:20.250041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:20.250041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:20.250041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:20.250041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:20.250041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:20.250041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:20.250041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:20.250041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:20.250041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:20.250041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:20.250041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:20.250041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:20.250041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:20.250898 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:20.250898 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:20.250898 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:20.250898 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:20.250898 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:20.250898 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:20.250898 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:20.250898 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:20.250898 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:20.250898 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:20.250898 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:20.250898 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:20.250898 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:20.250898 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:20.250898 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:20.250898 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:20.251679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:20.251679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:20.251679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:20.251679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:20.251679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:20.251679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:20.251679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:20.251679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:20.251679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:20.251679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:20.251679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 23:30:20.251679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:30:20.252325 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef336c05-170b-411d-9ed0-93daa0429f56 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:20.263492 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 111/224] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 17 23:30:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:20.263492 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5f1f8-086b-4b35-a9fd-f5b6a486c1eb None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:20.279066 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 114/225] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:30:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:20.292934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef336c05-170b-411d-9ed0-93daa0429f56 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:20.381281 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5f1f8-086b-4b35-a9fd-f5b6a486c1eb None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:20.426468 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da3b68d-17e1-48d4-aa60-072fe260a077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:20.426468 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3b68d-17e1-48d4-aa60-072fe260a077 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:20.426468 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3b68d-17e1-48d4-aa60-072fe260a077 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:20.426468 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3b68d-17e1-48d4-aa60-072fe260a077 None admin] PATH_INFO: `/v3/projects/e959a87f43754d66b9bd795c08d2d827` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:20.428076 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da3b68d-17e1-48d4-aa60-072fe260a077 None admin] RBAC: Authorizing `identity:get_project(project_id=e959a87f43754d66b9bd795c08d2d827)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:20.430220 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da3b68d-17e1-48d4-aa60-072fe260a077 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:20.438550 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 112/226] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:20 2022] GET /identity/v3/projects/e959a87f43754d66b9bd795c08d2d827 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:20.438550 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da3b68d-17e1-48d4-aa60-072fe260a077 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:20.491956 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:20.571918 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da3b68d-17e1-48d4-aa60-072fe260a077 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:20.616172 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8d4b1-0f54-469b-9b95-2ada799b88e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:20.617298 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8d4b1-0f54-469b-9b95-2ada799b88e1 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:20.617783 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8d4b1-0f54-469b-9b95-2ada799b88e1 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:20.618276 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8d4b1-0f54-469b-9b95-2ada799b88e1 None admin] PATH_INFO: `/v3/users/6530156892a74973822b17cdc23f6faa` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:20.619638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e8d4b1-0f54-469b-9b95-2ada799b88e1 None admin] RBAC: Authorizing `identity:get_user(user_id=6530156892a74973822b17cdc23f6faa)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:20.621313 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e8d4b1-0f54-469b-9b95-2ada799b88e1 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:20.623158 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 113/227] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:30:20 2022] GET /identity/v3/users/6530156892a74973822b17cdc23f6faa => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:20.624153 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8d4b1-0f54-469b-9b95-2ada799b88e1 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:20.639002 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef336c05-170b-411d-9ed0-93daa0429f56 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:20.744763 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8d4b1-0f54-469b-9b95-2ada799b88e1 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:20.750673 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:20.768229 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35713979-8cbd-4426-a5af-206baec590f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:20.768853 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35713979-8cbd-4426-a5af-206baec590f0 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:20.769132 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35713979-8cbd-4426-a5af-206baec590f0 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:20.769403 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35713979-8cbd-4426-a5af-206baec590f0 None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:20.770083 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35713979-8cbd-4426-a5af-206baec590f0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:20.777386 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35713979-8cbd-4426-a5af-206baec590f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:20.779249 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35713979-8cbd-4426-a5af-206baec590f0 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:20.789234 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffcc183-e21d-4070-9076-3520301872e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:20.790113 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffcc183-e21d-4070-9076-3520301872e9 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:20.790636 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffcc183-e21d-4070-9076-3520301872e9 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:20.791107 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffcc183-e21d-4070-9076-3520301872e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:20.792354 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffcc183-e21d-4070-9076-3520301872e9 None admin] RBAC: Adding query filter params (role.id=1182317a465349478dde4e28cb7cf96f, scope.project.id=e959a87f43754d66b9bd795c08d2d827, user.id=6dfd2578b49e454fa81bb5d1b9ae3b72) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:20.792871 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffcc183-e21d-4070-9076-3520301872e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:20.795533 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffcc183-e21d-4070-9076-3520301872e9 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:20.802335 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 114/228] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:30:20 2022] GET /identity/v3/role_assignments?user.id=6dfd2578b49e454fa81bb5d1b9ae3b72&scope.project.id=e959a87f43754d66b9bd795c08d2d827&role.id=1182317a465349478dde4e28cb7cf96f => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:20.803328 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffcc183-e21d-4070-9076-3520301872e9 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:20.812499 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 115/229] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:30:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 528 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:20.813493 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35713979-8cbd-4426-a5af-206baec590f0 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:20.959988 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:21.116050 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffcc183-e21d-4070-9076-3520301872e9 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:21.160002 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c80005-1536-4dc0-b86e-495edcb2ab9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:21.163982 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c80005-1536-4dc0-b86e-495edcb2ab9a None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:21.163982 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c80005-1536-4dc0-b86e-495edcb2ab9a None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:21.163982 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c80005-1536-4dc0-b86e-495edcb2ab9a None admin] PATH_INFO: `/v3/projects/e959a87f43754d66b9bd795c08d2d827` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:21.163982 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c80005-1536-4dc0-b86e-495edcb2ab9a None admin] RBAC: Authorizing `identity:get_project(project_id=e959a87f43754d66b9bd795c08d2d827)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:21.163982 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c80005-1536-4dc0-b86e-495edcb2ab9a None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:21.166550 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 115/230] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:20 2022] GET /identity/v3/projects/e959a87f43754d66b9bd795c08d2d827 => generated 302 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:21.175980 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c80005-1536-4dc0-b86e-495edcb2ab9a None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:21.224424 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35713979-8cbd-4426-a5af-206baec590f0 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:21.263033 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bdf376-28c9-46a6-9a2e-02f46ba8d68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:21.263669 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bdf376-28c9-46a6-9a2e-02f46ba8d68f None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:21.263950 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bdf376-28c9-46a6-9a2e-02f46ba8d68f None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:21.264261 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bdf376-28c9-46a6-9a2e-02f46ba8d68f None admin] PATH_INFO: `/v3/users/6530156892a74973822b17cdc23f6faa` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:21.265401 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bdf376-28c9-46a6-9a2e-02f46ba8d68f None admin] RBAC: Authorizing `identity:get_user(user_id=6530156892a74973822b17cdc23f6faa)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:21.266878 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bdf376-28c9-46a6-9a2e-02f46ba8d68f None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:21.268897 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 116/231] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:30:20 2022] GET /identity/v3/users/6530156892a74973822b17cdc23f6faa => generated 283 bytes in 455 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:21.276951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bdf376-28c9-46a6-9a2e-02f46ba8d68f None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:21.292425 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:21.315239 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c80005-1536-4dc0-b86e-495edcb2ab9a None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:21.386069 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e473c37-19b6-4fb4-94d1-a4b4e8b6fe62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:21.387090 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e473c37-19b6-4fb4-94d1-a4b4e8b6fe62 None admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:21.390822 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e473c37-19b6-4fb4-94d1-a4b4e8b6fe62 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:21.391315 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e473c37-19b6-4fb4-94d1-a4b4e8b6fe62 None admin] PATH_INFO: `/v3/projects/e959a87f43754d66b9bd795c08d2d827/users/6530156892a74973822b17cdc23f6faa/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:21.394826 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e473c37-19b6-4fb4-94d1-a4b4e8b6fe62 None admin] RBAC: Authorizing `identity:create_grant(project_id=e959a87f43754d66b9bd795c08d2d827, user_id=6530156892a74973822b17cdc23f6faa, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:21.396545 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e473c37-19b6-4fb4-94d1-a4b4e8b6fe62 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:21.423406 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 116/232] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 17 23:30:21 2022] PUT /identity/v3/projects/e959a87f43754d66b9bd795c08d2d827/users/6530156892a74973822b17cdc23f6faa/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:30:21.458018 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:21.468130 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bdf376-28c9-46a6-9a2e-02f46ba8d68f None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:21.507517 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d89c1-5e59-4cfa-9974-b0e91335139c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:21.508419 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d89c1-5e59-4cfa-9974-b0e91335139c None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:21.508901 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d89c1-5e59-4cfa-9974-b0e91335139c None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:21.509296 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d89c1-5e59-4cfa-9974-b0e91335139c None admin] PATH_INFO: `/v3/projects/e959a87f43754d66b9bd795c08d2d827` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:21.510592 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194d89c1-5e59-4cfa-9974-b0e91335139c None admin] RBAC: Authorizing `identity:get_project(project_id=e959a87f43754d66b9bd795c08d2d827)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:21.512335 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194d89c1-5e59-4cfa-9974-b0e91335139c None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:21.515307 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 117/233] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:21 2022] GET /identity/v3/projects/e959a87f43754d66b9bd795c08d2d827 => generated 302 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:21.525156 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e473c37-19b6-4fb4-94d1-a4b4e8b6fe62 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:21.543139 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f11c4b-7bac-492d-a2c8-3073a7ac0193 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:21.545245 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f11c4b-7bac-492d-a2c8-3073a7ac0193 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:21.546242 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f11c4b-7bac-492d-a2c8-3073a7ac0193 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:21.549893 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 118/234] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:21 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:21.555930 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac06193-dc21-4c2a-a35f-1080d70b43fa None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:21.559158 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac06193-dc21-4c2a-a35f-1080d70b43fa None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:21.559158 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac06193-dc21-4c2a-a35f-1080d70b43fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:21.597856 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-aac06193-dc21-4c2a-a35f-1080d70b43fa None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:21.601126 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-aac06193-dc21-4c2a-a35f-1080d70b43fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:21.723037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:21.748947 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e473c37-19b6-4fb4-94d1-a4b4e8b6fe62 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:21.760132 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 119/235] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 23:30:21.765971 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d88428-47ac-48ee-98b3-cfae55d8353c None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:21.767194 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d88428-47ac-48ee-98b3-cfae55d8353c None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:21.768560 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d88428-47ac-48ee-98b3-cfae55d8353c None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:21.779994 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 120/236] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:21 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:21.787985 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0803d2-111c-4876-a24e-2b382be19ca7 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:21.787985 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0803d2-111c-4876-a24e-2b382be19ca7 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:21.791990 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0803d2-111c-4876-a24e-2b382be19ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:21.851987 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-ac0803d2-111c-4876-a24e-2b382be19ca7 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:21.851987 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-ac0803d2-111c-4876-a24e-2b382be19ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:21.867181 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9031441-6514-427e-9a76-3cf05ad34ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:21.867976 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9031441-6514-427e-9a76-3cf05ad34ac8 None admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:21.868258 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9031441-6514-427e-9a76-3cf05ad34ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:21.868529 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9031441-6514-427e-9a76-3cf05ad34ac8 None admin] PATH_INFO: `/v3/projects/e959a87f43754d66b9bd795c08d2d827/users/6530156892a74973822b17cdc23f6faa/roles/3428ff8e92274de6817b7e8ee9a56d13` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:21.880868 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9031441-6514-427e-9a76-3cf05ad34ac8 None admin] RBAC: Authorizing `identity:create_grant(project_id=e959a87f43754d66b9bd795c08d2d827, user_id=6530156892a74973822b17cdc23f6faa, role_id=3428ff8e92274de6817b7e8ee9a56d13)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:21.882474 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9031441-6514-427e-9a76-3cf05ad34ac8 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:21.939987 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 117/237] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 17 23:30:21 2022] PUT /identity/v3/projects/e959a87f43754d66b9bd795c08d2d827/users/6530156892a74973822b17cdc23f6faa/roles/3428ff8e92274de6817b7e8ee9a56d13 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:30:21.962602 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 121/238] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:22.115977 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9031441-6514-427e-9a76-3cf05ad34ac8 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:22.236116 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9031441-6514-427e-9a76-3cf05ad34ac8 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:22.266499 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0803d2-111c-4876-a24e-2b382be19ca7 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:22.275935 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec01cf75-93e2-4f78-b77e-16736d95add9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:22.276636 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01cf75-93e2-4f78-b77e-16736d95add9 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:22.276916 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01cf75-93e2-4f78-b77e-16736d95add9 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:22.277187 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01cf75-93e2-4f78-b77e-16736d95add9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:22.298674 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec01cf75-93e2-4f78-b77e-16736d95add9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:22.304011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-ec01cf75-93e2-4f78-b77e-16736d95add9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 23:30:22.304011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:22.304011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:22.304011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:22.304011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:22.304011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:22.304011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:22.304011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:22.304011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:22.304011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:22.304011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:22.304011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:22.304011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:22.304011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:22.304011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:22.304011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:22.304889 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:22.304889 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:22.304889 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:22.304889 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:22.304889 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:22.304889 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:22.304889 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:22.304889 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:22.304889 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:22.304889 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:22.304889 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:22.304889 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:22.304889 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:22.304889 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:22.304889 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:22.304889 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:22.305687 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:22.305687 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:22.305687 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:22.305687 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:22.305687 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:22.305687 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:22.305687 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:22.305687 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:22.305687 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:22.305687 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:22.305687 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 23:30:22.305687 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:30:22.305687 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 118/239] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:30:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:22.316193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec01cf75-93e2-4f78-b77e-16736d95add9 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:22.316388 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:22.350526 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:22.460565 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0803d2-111c-4876-a24e-2b382be19ca7 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:22.501235 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec01cf75-93e2-4f78-b77e-16736d95add9 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:22.523954 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcbba04-c68b-4181-a73e-5d44ff7734d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:22.524608 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcbba04-c68b-4181-a73e-5d44ff7734d9 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:22.524900 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcbba04-c68b-4181-a73e-5d44ff7734d9 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:22.525167 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcbba04-c68b-4181-a73e-5d44ff7734d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:22.527412 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcbba04-c68b-4181-a73e-5d44ff7734d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:22.529736 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcbba04-c68b-4181-a73e-5d44ff7734d9 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:22.535266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 122/240] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:30:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:30:22.537501 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcbba04-c68b-4181-a73e-5d44ff7734d9 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:22.561728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c53072-f59a-4a0c-945d-21af4ed4997a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:22.562424 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c53072-f59a-4a0c-945d-21af4ed4997a None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:22.562766 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c53072-f59a-4a0c-945d-21af4ed4997a None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:22.564398 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c53072-f59a-4a0c-945d-21af4ed4997a None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:22.565149 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c53072-f59a-4a0c-945d-21af4ed4997a None admin] RBAC: Adding query filter params (name=reader) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:22.565513 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c53072-f59a-4a0c-945d-21af4ed4997a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:22.567152 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c53072-f59a-4a0c-945d-21af4ed4997a None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:22.582664 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 119/241] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 17 23:30:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:22.599073 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c53072-f59a-4a0c-945d-21af4ed4997a None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:22.717742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:22.757497 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c53072-f59a-4a0c-945d-21af4ed4997a None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:22.760547 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:22.778831 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcbba04-c68b-4181-a73e-5d44ff7734d9 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:22.819993 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20911fad-15ad-4367-a41f-a0dbe0de2d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:22.819993 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20911fad-15ad-4367-a41f-a0dbe0de2d63 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:22.819993 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20911fad-15ad-4367-a41f-a0dbe0de2d63 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:22.819993 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20911fad-15ad-4367-a41f-a0dbe0de2d63 None admin] PATH_INFO: `/v3/users/fbac01ecde4f45afa6619ea7c526f440` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:22.842879 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20911fad-15ad-4367-a41f-a0dbe0de2d63 None admin] RBAC: Authorizing `identity:get_user(user_id=fbac01ecde4f45afa6619ea7c526f440)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:22.844391 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20911fad-15ad-4367-a41f-a0dbe0de2d63 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:22.850754 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 120/242] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:30:22 2022] GET /identity/v3/users/fbac01ecde4f45afa6619ea7c526f440 => generated 297 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:22.851531 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20911fad-15ad-4367-a41f-a0dbe0de2d63 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:22.860249 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5dcd78-4623-4327-ae04-2a3f0ad4f78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:22.864760 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5dcd78-4623-4327-ae04-2a3f0ad4f78a None admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:22.865047 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5dcd78-4623-4327-ae04-2a3f0ad4f78a None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:22.865321 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5dcd78-4623-4327-ae04-2a3f0ad4f78a None admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:22.865983 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5dcd78-4623-4327-ae04-2a3f0ad4f78a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:22.867444 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5dcd78-4623-4327-ae04-2a3f0ad4f78a None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:22.909429 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 123/243] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Aug 17 23:30:22 2022] POST /identity/v3/users => generated 293 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:30:23.060287 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20911fad-15ad-4367-a41f-a0dbe0de2d63 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:23.112395 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf09d164-50eb-436a-96f3-171a0d649296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:23.113025 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09d164-50eb-436a-96f3-171a0d649296 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:23.113305 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09d164-50eb-436a-96f3-171a0d649296 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:23.113576 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09d164-50eb-436a-96f3-171a0d649296 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:23.118250 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:23.204101 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf09d164-50eb-436a-96f3-171a0d649296 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:23.205586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf09d164-50eb-436a-96f3-171a0d649296 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:23.280264 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a683511-cb53-48c5-ba93-8d4bf010dfe6 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:23.280888 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a683511-cb53-48c5-ba93-8d4bf010dfe6 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:23.281360 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a683511-cb53-48c5-ba93-8d4bf010dfe6 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:23.282910 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 124/244] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 23:30:23.288173 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d337c8-ed58-4f31-80e9-6e62282fc40c None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:23.288727 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d337c8-ed58-4f31-80e9-6e62282fc40c None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:23.289195 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d337c8-ed58-4f31-80e9-6e62282fc40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:23.310981 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-72d337c8-ed58-4f31-80e9-6e62282fc40c None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:23.312698 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-72d337c8-ed58-4f31-80e9-6e62282fc40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:23.320995 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 121/245] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:30:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:30:23.327282 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf09d164-50eb-436a-96f3-171a0d649296 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:23.370090 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:23.422768 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 125/246] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 23:30:23.446562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf09d164-50eb-436a-96f3-171a0d649296 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:23.493178 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bbf2c7-55de-4d9f-a979-3ea7c1807444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:23.493816 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbf2c7-55de-4d9f-a979-3ea7c1807444 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:23.494096 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbf2c7-55de-4d9f-a979-3ea7c1807444 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:23.494367 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbf2c7-55de-4d9f-a979-3ea7c1807444 None admin] PATH_INFO: `/v3/projects/e959a87f43754d66b9bd795c08d2d827` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:23.495519 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bbf2c7-55de-4d9f-a979-3ea7c1807444 None admin] RBAC: Authorizing `identity:get_project(project_id=e959a87f43754d66b9bd795c08d2d827)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:23.497070 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bbf2c7-55de-4d9f-a979-3ea7c1807444 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:23.498925 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 122/247] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:23 2022] GET /identity/v3/projects/e959a87f43754d66b9bd795c08d2d827 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:23.506497 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d337c8-ed58-4f31-80e9-6e62282fc40c None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:23.587880 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:23.618635 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d337c8-ed58-4f31-80e9-6e62282fc40c None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:23.657191 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e1500-8f09-4429-927b-6e234dd10281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:23.658257 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e1500-8f09-4429-927b-6e234dd10281 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:23.658792 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e1500-8f09-4429-927b-6e234dd10281 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:23.659286 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e1500-8f09-4429-927b-6e234dd10281 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:23.660308 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0e1500-8f09-4429-927b-6e234dd10281 None admin] RBAC: Adding query filter params (role.id=df3442912b3d43f0a36ebb936b3cb102, scope.project.id=e959a87f43754d66b9bd795c08d2d827, user.id=fbac01ecde4f45afa6619ea7c526f440) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:23.660814 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0e1500-8f09-4429-927b-6e234dd10281 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:23.662482 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0e1500-8f09-4429-927b-6e234dd10281 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:23.669582 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 126/248] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:30:23 2022] GET /identity/v3/role_assignments?user.id=fbac01ecde4f45afa6619ea7c526f440&scope.project.id=e959a87f43754d66b9bd795c08d2d827&role.id=df3442912b3d43f0a36ebb936b3cb102 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:23.701705 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba13021-092e-40d6-8187-a039c90b43f1 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:23.702313 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba13021-092e-40d6-8187-a039c90b43f1 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:23.702787 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba13021-092e-40d6-8187-a039c90b43f1 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:23.704780 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 123/249] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:23.711360 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2406b11-9ce5-4c9f-a2dc-79325b02a751 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:23.711951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2406b11-9ce5-4c9f-a2dc-79325b02a751 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:23.712457 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2406b11-9ce5-4c9f-a2dc-79325b02a751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:23.737971 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-f2406b11-9ce5-4c9f-a2dc-79325b02a751 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:23.739553 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-f2406b11-9ce5-4c9f-a2dc-79325b02a751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:23.769059 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba13021-092e-40d6-8187-a039c90b43f1 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:23.827295 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 127/250] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:23.924872 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba13021-092e-40d6-8187-a039c90b43f1 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:23.938083 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:24.014089 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a55a5-550b-4f97-a521-e9829e3aa32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:24.014749 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a55a5-550b-4f97-a521-e9829e3aa32c None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:24.015031 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a55a5-550b-4f97-a521-e9829e3aa32c None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:24.015306 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a55a5-550b-4f97-a521-e9829e3aa32c None admin] PATH_INFO: `/v3/roles/member` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:24.039176 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4a55a5-550b-4f97-a521-e9829e3aa32c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:24.044589 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-ea4a55a5-550b-4f97-a521-e9829e3aa32c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 23:30:24.044589 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:24.044589 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:24.044589 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:24.044589 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:24.044589 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:24.044589 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:24.044589 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:24.044589 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:24.044589 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:24.044589 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:24.044589 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:24.044589 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:24.044589 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:24.044589 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:24.044589 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:24.045454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:24.045454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:24.045454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:24.045454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:24.045454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:24.045454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:24.045454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:24.045454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:24.045454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:24.045454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:24.045454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:24.045454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:24.045454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:24.045454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:24.045454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:24.045454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:24.046272 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:24.046272 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:24.046272 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:24.046272 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:24.046272 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:24.046272 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:24.046272 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:24.046272 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:24.046272 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:24.046272 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:24.046272 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 23:30:24.046272 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:30:24.053073 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2406b11-9ce5-4c9f-a2dc-79325b02a751 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:24.055049 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 124/251] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:30:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:24.140320 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a55a5-550b-4f97-a521-e9829e3aa32c None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:24.191554 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:24.258090 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2406b11-9ce5-4c9f-a2dc-79325b02a751 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:24.327987 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8832b26-3648-4991-8040-baf88526f8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:24.327987 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8832b26-3648-4991-8040-baf88526f8ce None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:24.327987 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8832b26-3648-4991-8040-baf88526f8ce None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:24.327987 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8832b26-3648-4991-8040-baf88526f8ce None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:24.328389 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8832b26-3648-4991-8040-baf88526f8ce None admin] RBAC: Adding query filter params (name=member) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:24.328389 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8832b26-3648-4991-8040-baf88526f8ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:24.329326 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8832b26-3648-4991-8040-baf88526f8ce None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:24.337945 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a55a5-550b-4f97-a521-e9829e3aa32c None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:24.350816 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 128/252] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 17 23:30:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:24.358216 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8832b26-3648-4991-8040-baf88526f8ce None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:24.404115 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435b07d8-917e-48c6-91ec-bbc2f823b252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:24.404748 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b07d8-917e-48c6-91ec-bbc2f823b252 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:24.405023 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b07d8-917e-48c6-91ec-bbc2f823b252 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:24.405292 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b07d8-917e-48c6-91ec-bbc2f823b252 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:24.429345 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435b07d8-917e-48c6-91ec-bbc2f823b252 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:24.434143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-435b07d8-917e-48c6-91ec-bbc2f823b252 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:30:24.434143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:24.434143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:24.434143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:24.434143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:24.434143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:24.434143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:24.434143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:24.434143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:24.434143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:24.434143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:24.434143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:24.434143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:24.434143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:24.434143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:24.434143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:24.435087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:24.435087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:24.435087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:24.435087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:24.435087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:24.435087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:24.435087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:24.435087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:24.435087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:24.435087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:24.435087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:24.435087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:24.435087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:24.435087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:24.435087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:24.435087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:24.435871 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:24.435871 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:24.435871 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:24.435871 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:24.435871 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:24.435871 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:24.435871 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:24.435871 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:24.435871 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:24.435871 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:24.435871 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:30:24.435871 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:30:24.435871 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 125/253] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 17 23:30:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:24.445228 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435b07d8-917e-48c6-91ec-bbc2f823b252 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:24.506699 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:24.565108 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8832b26-3648-4991-8040-baf88526f8ce None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:24.624423 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:24.635768 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7cb0ec-1032-4763-aab0-a86b9a420627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:24.636420 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7cb0ec-1032-4763-aab0-a86b9a420627 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:24.636695 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7cb0ec-1032-4763-aab0-a86b9a420627 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:24.636963 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7cb0ec-1032-4763-aab0-a86b9a420627 None admin] PATH_INFO: `/v3/users/6dfd2578b49e454fa81bb5d1b9ae3b72` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:24.639463 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7cb0ec-1032-4763-aab0-a86b9a420627 None admin] RBAC: Authorizing `identity:get_user(user_id=6dfd2578b49e454fa81bb5d1b9ae3b72)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:24.640959 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7cb0ec-1032-4763-aab0-a86b9a420627 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:24.645443 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 129/254] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:30:24 2022] GET /identity/v3/users/6dfd2578b49e454fa81bb5d1b9ae3b72 => generated 297 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:24.646586 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1e93f-0da1-4626-afa8-7959808de669 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:24.646861 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1e93f-0da1-4626-afa8-7959808de669 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:24.647133 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1e93f-0da1-4626-afa8-7959808de669 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:24.650132 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435b07d8-917e-48c6-91ec-bbc2f823b252 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:24.654409 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 130/255] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:24.655211 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d1e93f-0da1-4626-afa8-7959808de669 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:24.755793 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156ce541-bbb1-4c15-b4f9-b334ba4600c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:24.756799 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ce541-bbb1-4c15-b4f9-b334ba4600c3 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:24.757296 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ce541-bbb1-4c15-b4f9-b334ba4600c3 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:24.757764 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ce541-bbb1-4c15-b4f9-b334ba4600c3 None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:24.758650 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156ce541-bbb1-4c15-b4f9-b334ba4600c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:24.759144 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156ce541-bbb1-4c15-b4f9-b334ba4600c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:24.768342 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d1e93f-0da1-4626-afa8-7959808de669 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:24.768704 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156ce541-bbb1-4c15-b4f9-b334ba4600c3 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:24.789682 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 126/256] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:30:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:24.791216 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19f4a5-7038-4802-be59-68af725c6b3a None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:24.791678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19f4a5-7038-4802-be59-68af725c6b3a None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:24.792157 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19f4a5-7038-4802-be59-68af725c6b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:24.810836 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e97af-f179-4383-8628-1f863e0e6012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:24.811598 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e97af-f179-4383-8628-1f863e0e6012 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:24.811947 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e97af-f179-4383-8628-1f863e0e6012 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:24.812364 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e97af-f179-4383-8628-1f863e0e6012 None admin] PATH_INFO: `/v3/projects/e959a87f43754d66b9bd795c08d2d827` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:24.819437 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e97af-f179-4383-8628-1f863e0e6012 None admin] RBAC: Authorizing `identity:get_project(project_id=e959a87f43754d66b9bd795c08d2d827)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:24.819437 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e97af-f179-4383-8628-1f863e0e6012 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:24.819437 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 131/257] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:24 2022] GET /identity/v3/projects/e959a87f43754d66b9bd795c08d2d827 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:24.819437 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e97af-f179-4383-8628-1f863e0e6012 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:24.829687 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-ea19f4a5-7038-4802-be59-68af725c6b3a None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:24.839995 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-ea19f4a5-7038-4802-be59-68af725c6b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:24.895720 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:24.939992 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e97af-f179-4383-8628-1f863e0e6012 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:24.974067 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45df1dc-c158-44c4-be78-74b6db872174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:24.974755 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45df1dc-c158-44c4-be78-74b6db872174 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:24.975033 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45df1dc-c158-44c4-be78-74b6db872174 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:24.975304 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45df1dc-c158-44c4-be78-74b6db872174 None admin] PATH_INFO: `/v3/users/6530156892a74973822b17cdc23f6faa` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:24.976505 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45df1dc-c158-44c4-be78-74b6db872174 None admin] RBAC: Authorizing `identity:get_user(user_id=6530156892a74973822b17cdc23f6faa)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:24.977959 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45df1dc-c158-44c4-be78-74b6db872174 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:24.979619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 132/258] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:30:24 2022] GET /identity/v3/users/6530156892a74973822b17cdc23f6faa => generated 283 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:24.980382 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45df1dc-c158-44c4-be78-74b6db872174 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:25.071626 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 127/259] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:25.072799 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea19f4a5-7038-4802-be59-68af725c6b3a None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:25.089030 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45df1dc-c158-44c4-be78-74b6db872174 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:25.117631 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:25.125090 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaabb2f-dd4d-4aac-95e1-b1f645601669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:25.125825 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaabb2f-dd4d-4aac-95e1-b1f645601669 None admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:25.126078 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaabb2f-dd4d-4aac-95e1-b1f645601669 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:25.126356 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaabb2f-dd4d-4aac-95e1-b1f645601669 None admin] PATH_INFO: `/v3/projects/e959a87f43754d66b9bd795c08d2d827/users/6dfd2578b49e454fa81bb5d1b9ae3b72/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:25.132468 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaabb2f-dd4d-4aac-95e1-b1f645601669 None admin] RBAC: Authorizing `identity:create_grant(project_id=e959a87f43754d66b9bd795c08d2d827, user_id=6dfd2578b49e454fa81bb5d1b9ae3b72, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:25.133933 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaabb2f-dd4d-4aac-95e1-b1f645601669 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:25.137051 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:25.156488 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 133/260] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 17 23:30:24 2022] PUT /identity/v3/projects/e959a87f43754d66b9bd795c08d2d827/users/6dfd2578b49e454fa81bb5d1b9ae3b72/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:30:25.286167 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea19f4a5-7038-4802-be59-68af725c6b3a None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:25.361776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfd610-e679-4c30-8895-7b9437bf1633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:25.362369 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfd610-e679-4c30-8895-7b9437bf1633 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:25.362650 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfd610-e679-4c30-8895-7b9437bf1633 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:25.362921 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfd610-e679-4c30-8895-7b9437bf1633 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:25.440467 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbfd610-e679-4c30-8895-7b9437bf1633 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:25.442031 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbfd610-e679-4c30-8895-7b9437bf1633 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:25.480328 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:25.601689 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 128/261] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:30:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:30:25.607349 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaabb2f-dd4d-4aac-95e1-b1f645601669 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:25.696643 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfd610-e679-4c30-8895-7b9437bf1633 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:25.839743 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaabb2f-dd4d-4aac-95e1-b1f645601669 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:25.880002 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfd610-e679-4c30-8895-7b9437bf1633 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:25.906310 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:25.909982 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af4e92-3207-41b6-af8c-93aacb51b8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:25.910669 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af4e92-3207-41b6-af8c-93aacb51b8b1 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:25.910945 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af4e92-3207-41b6-af8c-93aacb51b8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:25.911219 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af4e92-3207-41b6-af8c-93aacb51b8b1 None admin] PATH_INFO: `/v3/projects/e959a87f43754d66b9bd795c08d2d827` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:25.913188 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1af4e92-3207-41b6-af8c-93aacb51b8b1 None admin] RBAC: Authorizing `identity:get_project(project_id=e959a87f43754d66b9bd795c08d2d827)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:25.914723 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1af4e92-3207-41b6-af8c-93aacb51b8b1 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:25.919384 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 134/262] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:25 2022] GET /identity/v3/projects/e959a87f43754d66b9bd795c08d2d827 => generated 302 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:25.920609 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85da74-0ccb-4b09-af55-7e6641dbf083 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:25.920892 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85da74-0ccb-4b09-af55-7e6641dbf083 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:25.921163 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85da74-0ccb-4b09-af55-7e6641dbf083 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:25.922543 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 135/263] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:25.927159 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a85da74-0ccb-4b09-af55-7e6641dbf083 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:25.948783 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1264bb24-4cf5-460d-9d24-2e7f3126b771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:25.949332 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264bb24-4cf5-460d-9d24-2e7f3126b771 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:25.949614 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264bb24-4cf5-460d-9d24-2e7f3126b771 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:25.949917 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264bb24-4cf5-460d-9d24-2e7f3126b771 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:25.985404 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1264bb24-4cf5-460d-9d24-2e7f3126b771 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:25.990740 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-1264bb24-4cf5-460d-9d24-2e7f3126b771 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 23:30:25.990740 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:25.990740 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:25.990740 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:25.990740 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:25.990740 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:25.990740 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:25.990740 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:25.990740 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:25.990740 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:25.990740 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:25.990740 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:25.990740 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:25.990740 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:25.990740 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:25.990740 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:25.991629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:25.991629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:25.991629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:25.991629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:25.991629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:25.991629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:25.991629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:25.991629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:25.991629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:25.991629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:25.991629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:25.991629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:25.991629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:25.991629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:25.991629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:25.991629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:25.992503 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:25.992503 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:25.992503 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:25.992503 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:25.992503 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:25.992503 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:25.992503 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:25.992503 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:25.992503 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:25.992503 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:25.992503 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 23:30:25.992503 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:30:25.995253 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 129/264] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 17 23:30:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:30:25.996722 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4450ae-4460-48cf-9fce-6b761db29bf3 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:25.997708 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4450ae-4460-48cf-9fce-6b761db29bf3 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:25.998162 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4450ae-4460-48cf-9fce-6b761db29bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:26.017179 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-6a4450ae-4460-48cf-9fce-6b761db29bf3 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:26.018740 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-6a4450ae-4460-48cf-9fce-6b761db29bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:26.024778 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:26.110816 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a85da74-0ccb-4b09-af55-7e6641dbf083 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:26.128719 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 130/265] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:26.130098 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4450ae-4460-48cf-9fce-6b761db29bf3 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:26.182569 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ec5ae-8b52-439e-a4d9-150bab935cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:26.183470 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ec5ae-8b52-439e-a4d9-150bab935cea None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:26.183924 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ec5ae-8b52-439e-a4d9-150bab935cea None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:26.185810 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ec5ae-8b52-439e-a4d9-150bab935cea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:26.186754 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0ec5ae-8b52-439e-a4d9-150bab935cea None admin] RBAC: Adding query filter params (role.id=33bd60d3b0e94dcbbfe97e87f91a8d78, scope.project.id=e959a87f43754d66b9bd795c08d2d827, user.id=6530156892a74973822b17cdc23f6faa) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:26.187281 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0ec5ae-8b52-439e-a4d9-150bab935cea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:26.189111 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0ec5ae-8b52-439e-a4d9-150bab935cea None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:26.203328 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 136/266] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:30:25 2022] GET /identity/v3/role_assignments?user.id=6530156892a74973822b17cdc23f6faa&scope.project.id=e959a87f43754d66b9bd795c08d2d827&role.id=33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 608 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:26.339953 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4450ae-4460-48cf-9fce-6b761db29bf3 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:26.352224 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:26.359322 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3760af-4872-4e06-af84-89682da76cd8 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:26.359922 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3760af-4872-4e06-af84-89682da76cd8 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:26.360582 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3760af-4872-4e06-af84-89682da76cd8 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:26.362135 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 137/267] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 23:30:26.368580 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d56192-6246-4f95-9fd5-a1331b01cb2b None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:26.369157 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d56192-6246-4f95-9fd5-a1331b01cb2b None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:26.369647 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d56192-6246-4f95-9fd5-a1331b01cb2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:26.392883 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-c6d56192-6246-4f95-9fd5-a1331b01cb2b None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:26.395096 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-c6d56192-6246-4f95-9fd5-a1331b01cb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:26.412567 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880aad3-7fe5-42f5-a974-250511d2e8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:26.413865 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880aad3-7fe5-42f5-a974-250511d2e8aa None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:26.414343 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880aad3-7fe5-42f5-a974-250511d2e8aa None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:26.414810 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880aad3-7fe5-42f5-a974-250511d2e8aa None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:26.415720 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b880aad3-7fe5-42f5-a974-250511d2e8aa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:26.416263 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b880aad3-7fe5-42f5-a974-250511d2e8aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:26.419772 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b880aad3-7fe5-42f5-a974-250511d2e8aa None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:26.435265 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 131/268] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:30:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:26.442296 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880aad3-7fe5-42f5-a974-250511d2e8aa None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:26.572004 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 138/269] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 23:30:26.573375 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d56192-6246-4f95-9fd5-a1331b01cb2b None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:26.606762 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880aad3-7fe5-42f5-a974-250511d2e8aa None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:26.653512 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:26.657536 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e8d4e-b677-416a-9c2c-d7a6b8a20131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:26.658459 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e8d4e-b677-416a-9c2c-d7a6b8a20131 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:26.658961 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e8d4e-b677-416a-9c2c-d7a6b8a20131 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:26.659423 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e8d4e-b677-416a-9c2c-d7a6b8a20131 None admin] PATH_INFO: `/v3/users/6530156892a74973822b17cdc23f6faa` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:26.663413 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5e8d4e-b677-416a-9c2c-d7a6b8a20131 None admin] RBAC: Authorizing `identity:get_user(user_id=6530156892a74973822b17cdc23f6faa)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:26.665148 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5e8d4e-b677-416a-9c2c-d7a6b8a20131 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:26.667427 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 132/270] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:30:26 2022] GET /identity/v3/users/6530156892a74973822b17cdc23f6faa => generated 283 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:26.669997 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc1ad3-279b-4a9d-9767-d0aba171ca29 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:26.670525 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc1ad3-279b-4a9d-9767-d0aba171ca29 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:26.670988 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc1ad3-279b-4a9d-9767-d0aba171ca29 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:26.672518 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 133/271] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:26.677561 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dc1ad3-279b-4a9d-9767-d0aba171ca29 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:26.706481 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d56192-6246-4f95-9fd5-a1331b01cb2b None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:26.725087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:26.745335 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a319d067-dd57-430a-b583-153db9a702b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:26.746236 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319d067-dd57-430a-b583-153db9a702b4 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:26.746731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319d067-dd57-430a-b583-153db9a702b4 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:26.748173 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319d067-dd57-430a-b583-153db9a702b4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:26.771949 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a319d067-dd57-430a-b583-153db9a702b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:26.776972 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-a319d067-dd57-430a-b583-153db9a702b4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 23:30:26.776972 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:26.776972 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:26.776972 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:26.776972 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:26.776972 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:26.776972 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:26.776972 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:26.776972 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:26.776972 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:26.776972 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:26.776972 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:26.776972 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:26.776972 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:26.776972 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:26.776972 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:26.778156 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:26.778156 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:26.778156 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:26.778156 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:26.778156 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:26.778156 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:26.778156 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:26.778156 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:26.778156 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:26.778156 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:26.778156 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:26.778156 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:26.778156 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:26.778156 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:26.778156 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:26.778156 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:26.778973 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:26.778973 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:26.778973 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:26.778973 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:26.778973 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:26.778973 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:26.778973 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:26.778973 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:26.778973 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:26.778973 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:26.778973 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 23:30:26.778973 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:30:26.783220 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 139/272] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:30:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:26.785023 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb0f50-d746-4a09-b015-693c43adf35f None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:26.785308 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb0f50-d746-4a09-b015-693c43adf35f None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:26.785581 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb0f50-d746-4a09-b015-693c43adf35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:26.796985 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dc1ad3-279b-4a9d-9767-d0aba171ca29 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:26.804022 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-6efb0f50-d746-4a09-b015-693c43adf35f None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:26.805321 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-6efb0f50-d746-4a09-b015-693c43adf35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:26.834748 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6244bf47-eb34-4f3c-a7ad-105c021c238c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:26.835681 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244bf47-eb34-4f3c-a7ad-105c021c238c None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:26.836204 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244bf47-eb34-4f3c-a7ad-105c021c238c None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:26.836687 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244bf47-eb34-4f3c-a7ad-105c021c238c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:26.899988 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 140/273] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:26.899988 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efb0f50-d746-4a09-b015-693c43adf35f None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:26.915103 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6244bf47-eb34-4f3c-a7ad-105c021c238c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:26.917402 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6244bf47-eb34-4f3c-a7ad-105c021c238c None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:26.942694 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:26.965320 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:27.007991 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efb0f50-d746-4a09-b015-693c43adf35f None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:27.045632 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0186ecd6-3676-4c91-a5c2-f1f786593b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:27.046652 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0186ecd6-3676-4c91-a5c2-f1f786593b6f None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:27.047198 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0186ecd6-3676-4c91-a5c2-f1f786593b6f None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:27.047477 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0186ecd6-3676-4c91-a5c2-f1f786593b6f None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:27.048432 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0186ecd6-3676-4c91-a5c2-f1f786593b6f None admin] RBAC: Adding query filter params (name=member) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:27.048741 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0186ecd6-3676-4c91-a5c2-f1f786593b6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:27.050906 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0186ecd6-3676-4c91-a5c2-f1f786593b6f None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:27.052736 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 134/274] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:30:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 17 23:30:27.053659 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6244bf47-eb34-4f3c-a7ad-105c021c238c None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:27.064472 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 141/275] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 17 23:30:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:27.069628 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0186ecd6-3676-4c91-a5c2-f1f786593b6f None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:27.181919 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0186ecd6-3676-4c91-a5c2-f1f786593b6f None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:27.182213 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6244bf47-eb34-4f3c-a7ad-105c021c238c None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:27.194655 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:27.223603 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61955e74-9643-4f24-aa5e-b7d769887ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:27.224525 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61955e74-9643-4f24-aa5e-b7d769887ffa None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:27.224939 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61955e74-9643-4f24-aa5e-b7d769887ffa None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:27.225346 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61955e74-9643-4f24-aa5e-b7d769887ffa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:27.225493 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a545bfce-0cc5-414e-a98f-141a42d5e979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:27.226142 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545bfce-0cc5-414e-a98f-141a42d5e979 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:27.226409 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545bfce-0cc5-414e-a98f-141a42d5e979 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:27.226665 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545bfce-0cc5-414e-a98f-141a42d5e979 None admin] PATH_INFO: `/v3/projects/e959a87f43754d66b9bd795c08d2d827` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:27.227761 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a545bfce-0cc5-414e-a98f-141a42d5e979 None admin] RBAC: Authorizing `identity:get_project(project_id=e959a87f43754d66b9bd795c08d2d827)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:27.229088 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:27.229282 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a545bfce-0cc5-414e-a98f-141a42d5e979 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:27.230942 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 142/276] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:27 2022] GET /identity/v3/projects/e959a87f43754d66b9bd795c08d2d827 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:27.232271 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a545bfce-0cc5-414e-a98f-141a42d5e979 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:27.246365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61955e74-9643-4f24-aa5e-b7d769887ffa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:27.252485 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-61955e74-9643-4f24-aa5e-b7d769887ffa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 23:30:27.252485 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:27.252485 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:27.252485 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:27.252485 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:27.252485 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:27.252485 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:27.252485 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:27.252485 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:27.252485 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:27.252485 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:27.252485 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:27.252485 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:27.252485 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:27.252485 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:27.252485 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:27.253521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:27.253521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:27.253521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:27.253521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:27.253521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:27.253521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:27.253521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:27.253521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:27.253521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:27.253521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:27.253521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:27.253521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:27.253521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:27.253521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:27.253521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:27.253521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:27.254472 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:27.254472 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:27.254472 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:27.254472 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:27.254472 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:27.254472 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:27.254472 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:27.254472 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:27.254472 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:27.254472 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:27.254472 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 23:30:27.254472 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:30:27.256527 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 135/277] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:30:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:27.257545 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61955e74-9643-4f24-aa5e-b7d769887ffa None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:27.341064 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a545bfce-0cc5-414e-a98f-141a42d5e979 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:27.365263 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61955e74-9643-4f24-aa5e-b7d769887ffa None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:27.374480 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16dde43-b00c-4b96-b424-a794120b9418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:27.375109 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16dde43-b00c-4b96-b424-a794120b9418 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:27.375387 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16dde43-b00c-4b96-b424-a794120b9418 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:27.375660 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16dde43-b00c-4b96-b424-a794120b9418 None admin] PATH_INFO: `/v3/users/0bbf609b8b15441d8ee7b89898880156` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:27.388682 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16dde43-b00c-4b96-b424-a794120b9418 None admin] RBAC: Authorizing `identity:get_user(user_id=0bbf609b8b15441d8ee7b89898880156)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:27.390139 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16dde43-b00c-4b96-b424-a794120b9418 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:27.391779 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 143/278] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:30:27 2022] GET /identity/v3/users/0bbf609b8b15441d8ee7b89898880156 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:27.392575 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16dde43-b00c-4b96-b424-a794120b9418 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:27.406468 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fbd3a1-d676-488d-a71e-428905cedce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:27.407342 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fbd3a1-d676-488d-a71e-428905cedce6 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:27.407814 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fbd3a1-d676-488d-a71e-428905cedce6 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:27.408797 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fbd3a1-d676-488d-a71e-428905cedce6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:27.410952 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fbd3a1-d676-488d-a71e-428905cedce6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:27.412719 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fbd3a1-d676-488d-a71e-428905cedce6 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:27.420145 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 136/279] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:30:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:27.421240 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fbd3a1-d676-488d-a71e-428905cedce6 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:27.430368 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:27.493701 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:27.509483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16dde43-b00c-4b96-b424-a794120b9418 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:27.534837 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fbd3a1-d676-488d-a71e-428905cedce6 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:27.545841 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00497f6e-f0de-4c5a-a2e5-5a98bf99f135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:27.546446 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00497f6e-f0de-4c5a-a2e5-5a98bf99f135 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:27.546724 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00497f6e-f0de-4c5a-a2e5-5a98bf99f135 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:27.546994 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00497f6e-f0de-4c5a-a2e5-5a98bf99f135 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:27.547722 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00497f6e-f0de-4c5a-a2e5-5a98bf99f135 None admin] RBAC: Adding query filter params (role.id=3428ff8e92274de6817b7e8ee9a56d13, scope.project.id=e959a87f43754d66b9bd795c08d2d827, user.id=6530156892a74973822b17cdc23f6faa) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:27.548073 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00497f6e-f0de-4c5a-a2e5-5a98bf99f135 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:27.549520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00497f6e-f0de-4c5a-a2e5-5a98bf99f135 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:27.564012 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 144/280] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:30:27 2022] GET /identity/v3/role_assignments?user.id=6530156892a74973822b17cdc23f6faa&scope.project.id=e959a87f43754d66b9bd795c08d2d827&role.id=3428ff8e92274de6817b7e8ee9a56d13 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:27.564012 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00497f6e-f0de-4c5a-a2e5-5a98bf99f135 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:27.570961 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d22200-9c9f-4d04-a2d7-7dd4b83b124b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:27.571572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d22200-9c9f-4d04-a2d7-7dd4b83b124b None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:27.571850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d22200-9c9f-4d04-a2d7-7dd4b83b124b None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:27.572151 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d22200-9c9f-4d04-a2d7-7dd4b83b124b None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:27.572833 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d22200-9c9f-4d04-a2d7-7dd4b83b124b None admin] RBAC: Adding query filter params (name=reader) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:27.573135 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d22200-9c9f-4d04-a2d7-7dd4b83b124b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:27.574567 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d22200-9c9f-4d04-a2d7-7dd4b83b124b None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:27.584317 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 137/281] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 17 23:30:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:27.585174 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d22200-9c9f-4d04-a2d7-7dd4b83b124b None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:27.703478 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:27.704344 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d22200-9c9f-4d04-a2d7-7dd4b83b124b None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:27.708005 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00497f6e-f0de-4c5a-a2e5-5a98bf99f135 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:27.738768 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:27.748524 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a33ae9-23f8-4688-98c5-8589ef730b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:27.749619 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a33ae9-23f8-4688-98c5-8589ef730b73 None admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:27.749619 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a33ae9-23f8-4688-98c5-8589ef730b73 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:27.749826 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a33ae9-23f8-4688-98c5-8589ef730b73 None admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:27.750323 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a33ae9-23f8-4688-98c5-8589ef730b73 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:27.751778 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a33ae9-23f8-4688-98c5-8589ef730b73 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:27.754629 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe391dcd-846f-45d2-b26e-d23ccd5090c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:27.755330 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe391dcd-846f-45d2-b26e-d23ccd5090c3 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:27.755679 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe391dcd-846f-45d2-b26e-d23ccd5090c3 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:27.755979 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe391dcd-846f-45d2-b26e-d23ccd5090c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:27.756748 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe391dcd-846f-45d2-b26e-d23ccd5090c3 None admin] RBAC: Adding query filter params (role.id=1182317a465349478dde4e28cb7cf96f, user.id=0bbf609b8b15441d8ee7b89898880156, scope.system=all) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:27.757053 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe391dcd-846f-45d2-b26e-d23ccd5090c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:27.758485 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe391dcd-846f-45d2-b26e-d23ccd5090c3 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:27.766523 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 145/282] 10.222.0.26 () {56 vars in 1308 bytes} [Wed Aug 17 23:30:27 2022] GET /identity/v3/role_assignments?user.id=0bbf609b8b15441d8ee7b89898880156&scope.system=all&role.id=1182317a465349478dde4e28cb7cf96f => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:27.767218 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe391dcd-846f-45d2-b26e-d23ccd5090c3 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:27.780159 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 138/283] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Aug 17 23:30:27 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:30:27.781654 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ffe74c-a013-4645-a54a-1e37a6c40b54 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:27.781932 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ffe74c-a013-4645-a54a-1e37a6c40b54 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:27.782203 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ffe74c-a013-4645-a54a-1e37a6c40b54 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:27.783644 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 139/284] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:27.792850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc883d-05fa-46be-beaf-082269479d2e None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:27.793134 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc883d-05fa-46be-beaf-082269479d2e None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:27.793413 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc883d-05fa-46be-beaf-082269479d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:27.829080 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-d3fc883d-05fa-46be-beaf-082269479d2e None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:27.832078 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-d3fc883d-05fa-46be-beaf-082269479d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:27.931992 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe391dcd-846f-45d2-b26e-d23ccd5090c3 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:27.932360 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 140/285] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:27.966218 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21aaa5-e151-41b4-842b-acc7ecee5cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:27.966779 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21aaa5-e151-41b4-842b-acc7ecee5cdb None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:27.967213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21aaa5-e151-41b4-842b-acc7ecee5cdb None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:27.967554 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21aaa5-e151-41b4-842b-acc7ecee5cdb None admin] PATH_INFO: `/v3/users/fbac01ecde4f45afa6619ea7c526f440` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:27.968798 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa21aaa5-e151-41b4-842b-acc7ecee5cdb None admin] RBAC: Authorizing `identity:get_user(user_id=fbac01ecde4f45afa6619ea7c526f440)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:27.970339 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa21aaa5-e151-41b4-842b-acc7ecee5cdb None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:27.972220 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 146/286] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:30:27 2022] GET /identity/v3/users/fbac01ecde4f45afa6619ea7c526f440 => generated 297 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:27.977671 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc883d-05fa-46be-beaf-082269479d2e None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:28.099987 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc883d-05fa-46be-beaf-082269479d2e None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:28.115392 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:28.143642 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327816cf-bd3d-4241-98b0-79952bed552b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:28.144324 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327816cf-bd3d-4241-98b0-79952bed552b None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:28.144609 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327816cf-bd3d-4241-98b0-79952bed552b None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:28.144887 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327816cf-bd3d-4241-98b0-79952bed552b None admin] PATH_INFO: `/v3/projects/e959a87f43754d66b9bd795c08d2d827` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:28.146021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327816cf-bd3d-4241-98b0-79952bed552b None admin] RBAC: Authorizing `identity:get_project(project_id=e959a87f43754d66b9bd795c08d2d827)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:28.147511 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327816cf-bd3d-4241-98b0-79952bed552b None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:28.149603 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 141/287] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:27 2022] GET /identity/v3/projects/e959a87f43754d66b9bd795c08d2d827 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:28.155528 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21aaa5-e151-41b4-842b-acc7ecee5cdb None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:28.193350 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:28.209340 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327816cf-bd3d-4241-98b0-79952bed552b None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:28.282004 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21aaa5-e151-41b4-842b-acc7ecee5cdb None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:28.353518 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327816cf-bd3d-4241-98b0-79952bed552b None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:28.354806 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ebc775-2246-40e9-8114-cfc4e1132f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:28.355586 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ebc775-2246-40e9-8114-cfc4e1132f96 None admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:28.355876 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ebc775-2246-40e9-8114-cfc4e1132f96 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:28.356702 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ebc775-2246-40e9-8114-cfc4e1132f96 None admin] PATH_INFO: `/v3/projects/e959a87f43754d66b9bd795c08d2d827/users/fbac01ecde4f45afa6619ea7c526f440/roles/df3442912b3d43f0a36ebb936b3cb102` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:28.358765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ebc775-2246-40e9-8114-cfc4e1132f96 None admin] RBAC: Authorizing `identity:create_grant(project_id=e959a87f43754d66b9bd795c08d2d827, user_id=fbac01ecde4f45afa6619ea7c526f440, role_id=df3442912b3d43f0a36ebb936b3cb102)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:28.360261 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ebc775-2246-40e9-8114-cfc4e1132f96 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:28.394188 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 147/288] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 17 23:30:28 2022] PUT /identity/v3/projects/e959a87f43754d66b9bd795c08d2d827/users/fbac01ecde4f45afa6619ea7c526f440/roles/df3442912b3d43f0a36ebb936b3cb102 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:30:28.401220 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac3997b-ce4f-419c-bcff-3c2f6f16b599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:28.402253 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3997b-ce4f-419c-bcff-3c2f6f16b599 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:28.403516 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3997b-ce4f-419c-bcff-3c2f6f16b599 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:28.404063 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3997b-ce4f-419c-bcff-3c2f6f16b599 None admin] PATH_INFO: `/v3/roles/member` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:28.416639 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac3997b-ce4f-419c-bcff-3c2f6f16b599 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:28.423454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-5ac3997b-ce4f-419c-bcff-3c2f6f16b599 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 23:30:28.423454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:28.423454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:28.423454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:28.423454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:28.423454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:28.423454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:28.423454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:28.423454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:28.423454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:28.423454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:28.423454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:28.423454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:28.423454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:28.423454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:28.423454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:28.424420 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:28.424420 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:28.424420 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:28.424420 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:28.424420 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:28.424420 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:28.424420 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:28.424420 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:28.424420 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:28.424420 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:28.424420 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:28.424420 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:28.424420 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:28.424420 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:28.424420 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:28.424420 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:28.425196 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:28.425196 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:28.425196 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:28.425196 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:28.425196 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:28.425196 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:28.425196 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:28.425196 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:28.425196 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:28.425196 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:28.425196 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 23:30:28.425196 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:30:28.425803 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 142/289] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:30:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:28.433348 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ebc775-2246-40e9-8114-cfc4e1132f96 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:28.602347 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:28.613529 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ebc775-2246-40e9-8114-cfc4e1132f96 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:28.658314 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13994764-0601-41e5-85a7-b243d9c89be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:28.658919 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13994764-0601-41e5-85a7-b243d9c89be4 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:28.659192 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13994764-0601-41e5-85a7-b243d9c89be4 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:28.659461 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13994764-0601-41e5-85a7-b243d9c89be4 None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:28.660174 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13994764-0601-41e5-85a7-b243d9c89be4 None admin] RBAC: Adding query filter params (name=member) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:28.660474 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13994764-0601-41e5-85a7-b243d9c89be4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:28.661868 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13994764-0601-41e5-85a7-b243d9c89be4 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:28.674668 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 148/290] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 17 23:30:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:28.680305 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac3997b-ce4f-419c-bcff-3c2f6f16b599 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:28.723787 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:28.812323 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac3997b-ce4f-419c-bcff-3c2f6f16b599 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:28.864877 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb6f42-28ae-4a23-ba72-922e5a6cc0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:28.865763 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb6f42-28ae-4a23-ba72-922e5a6cc0d3 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:28.866187 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb6f42-28ae-4a23-ba72-922e5a6cc0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:28.866598 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb6f42-28ae-4a23-ba72-922e5a6cc0d3 None admin] PATH_INFO: `/v3/users/6dfd2578b49e454fa81bb5d1b9ae3b72` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:28.870743 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fb6f42-28ae-4a23-ba72-922e5a6cc0d3 None admin] RBAC: Authorizing `identity:get_user(user_id=6dfd2578b49e454fa81bb5d1b9ae3b72)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:28.873003 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fb6f42-28ae-4a23-ba72-922e5a6cc0d3 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:28.875060 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 143/291] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:30:28 2022] GET /identity/v3/users/6dfd2578b49e454fa81bb5d1b9ae3b72 => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:28.880781 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13994764-0601-41e5-85a7-b243d9c89be4 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:29.074742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13994764-0601-41e5-85a7-b243d9c89be4 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:29.144754 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c331c8fc-28ce-49ed-872a-1f38ea6954ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:29.145290 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c331c8fc-28ce-49ed-872a-1f38ea6954ab None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:29.145573 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c331c8fc-28ce-49ed-872a-1f38ea6954ab None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:29.145839 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c331c8fc-28ce-49ed-872a-1f38ea6954ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:29.177622 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:29.271220 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c331c8fc-28ce-49ed-872a-1f38ea6954ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:29.272706 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c331c8fc-28ce-49ed-872a-1f38ea6954ab None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:29.453149 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 149/292] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:30:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:30:29.459003 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb6f42-28ae-4a23-ba72-922e5a6cc0d3 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:29.575893 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb6f42-28ae-4a23-ba72-922e5a6cc0d3 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:29.585895 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:29.614338 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebede67f-1c31-495b-8d7c-3e998f5d8b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:29.615280 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebede67f-1c31-495b-8d7c-3e998f5d8b65 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:29.615702 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebede67f-1c31-495b-8d7c-3e998f5d8b65 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:29.616150 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebede67f-1c31-495b-8d7c-3e998f5d8b65 None admin] PATH_INFO: `/v3/projects/e959a87f43754d66b9bd795c08d2d827` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:29.617530 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebede67f-1c31-495b-8d7c-3e998f5d8b65 None admin] RBAC: Authorizing `identity:get_project(project_id=e959a87f43754d66b9bd795c08d2d827)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:29.619136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebede67f-1c31-495b-8d7c-3e998f5d8b65 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:29.621385 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 144/293] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:29 2022] GET /identity/v3/projects/e959a87f43754d66b9bd795c08d2d827 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:29.629143 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c331c8fc-28ce-49ed-872a-1f38ea6954ab None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:29.723829 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:29.820693 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c331c8fc-28ce-49ed-872a-1f38ea6954ab None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:29.888665 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee603b59-ee84-464f-b17b-3f97bac419ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:29.889278 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee603b59-ee84-464f-b17b-3f97bac419ac None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:29.889556 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee603b59-ee84-464f-b17b-3f97bac419ac None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:29.889826 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee603b59-ee84-464f-b17b-3f97bac419ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:29.890550 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee603b59-ee84-464f-b17b-3f97bac419ac None admin] RBAC: Adding query filter params (role.id=1182317a465349478dde4e28cb7cf96f, scope.project.id=e959a87f43754d66b9bd795c08d2d827, user.id=6dfd2578b49e454fa81bb5d1b9ae3b72) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:29.890853 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee603b59-ee84-464f-b17b-3f97bac419ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:29.892314 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee603b59-ee84-464f-b17b-3f97bac419ac None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:29.898738 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 150/294] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:30:29 2022] GET /identity/v3/role_assignments?user.id=6dfd2578b49e454fa81bb5d1b9ae3b72&scope.project.id=e959a87f43754d66b9bd795c08d2d827&role.id=1182317a465349478dde4e28cb7cf96f => generated 608 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:29.942073 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3070c1-9256-4a25-9315-e8e0adb42a6e None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:29.942594 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3070c1-9256-4a25-9315-e8e0adb42a6e None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:29.943015 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3070c1-9256-4a25-9315-e8e0adb42a6e None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:29.945011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 145/295] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:29.951057 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfcf08-6066-49ce-8a63-e25b90899923 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:29.951337 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfcf08-6066-49ce-8a63-e25b90899923 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:29.951603 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfcf08-6066-49ce-8a63-e25b90899923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:29.989184 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-e8dfcf08-6066-49ce-8a63-e25b90899923 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:29.990314 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-e8dfcf08-6066-49ce-8a63-e25b90899923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:30.082439 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 151/296] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:30.384256 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3070c1-9256-4a25-9315-e8e0adb42a6e None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:30.503769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3070c1-9256-4a25-9315-e8e0adb42a6e None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:30.509121 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576ed7f-2eea-464c-afc1-50ee5f22b4f0 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:30.509401 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576ed7f-2eea-464c-afc1-50ee5f22b4f0 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:30.509672 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576ed7f-2eea-464c-afc1-50ee5f22b4f0 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:30.511006 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 152/297] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 23:30:30.516204 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fea64-7535-4f7f-b420-a31dc7d8fcaa None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:30.516489 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fea64-7535-4f7f-b420-a31dc7d8fcaa None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:30.516767 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fea64-7535-4f7f-b420-a31dc7d8fcaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:30.554636 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b4f3e-982c-4f11-9772-d14994f5a08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:30.555272 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b4f3e-982c-4f11-9772-d14994f5a08a None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:30.555547 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b4f3e-982c-4f11-9772-d14994f5a08a None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:30.555815 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b4f3e-982c-4f11-9772-d14994f5a08a None admin] PATH_INFO: `/v3/roles/member` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:30.563446 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-5a7fea64-7535-4f7f-b420-a31dc7d8fcaa None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:30.566648 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-5a7fea64-7535-4f7f-b420-a31dc7d8fcaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:30.579230 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2b4f3e-982c-4f11-9772-d14994f5a08a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:30.584769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-4e2b4f3e-982c-4f11-9772-d14994f5a08a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 23:30:30.584769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:30.584769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:30.584769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:30.584769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:30.584769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:30.584769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:30.584769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:30.584769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:30.584769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:30.584769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:30.584769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:30.584769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:30.584769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:30.584769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:30.584769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:30.585685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:30.585685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:30.585685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:30.585685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:30.585685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:30.585685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:30.585685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:30.585685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:30.585685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:30.585685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:30.585685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:30.585685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:30.585685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:30.585685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:30.585685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:30.585685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:30.586457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:30.586457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:30.586457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:30.586457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:30.586457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:30.586457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:30.586457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:30.586457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:30.586457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:30.586457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:30.586457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 23:30:30.586457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:30:30.588796 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 146/298] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:30:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:30.594698 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b4f3e-982c-4f11-9772-d14994f5a08a None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:30.620843 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:30.635830 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:30.647601 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 153/299] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:30.651007 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9121448c-854b-4757-9848-15598bbb5c9d None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:30.651290 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9121448c-854b-4757-9848-15598bbb5c9d None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:30.651558 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9121448c-854b-4757-9848-15598bbb5c9d None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:30.653816 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 154/300] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:30.661482 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7126f-0994-4bca-82ac-2db9d4de1f4c None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:30.663557 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7126f-0994-4bca-82ac-2db9d4de1f4c None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:30.663841 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7126f-0994-4bca-82ac-2db9d4de1f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:30.692490 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-f3a7126f-0994-4bca-82ac-2db9d4de1f4c None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:30.694732 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-f3a7126f-0994-4bca-82ac-2db9d4de1f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:30.714855 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b4f3e-982c-4f11-9772-d14994f5a08a None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:30.751794 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a166dd-09a3-4316-9467-585eb39b0d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:30.752430 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a166dd-09a3-4316-9467-585eb39b0d56 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:30.752712 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a166dd-09a3-4316-9467-585eb39b0d56 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:30.752982 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a166dd-09a3-4316-9467-585eb39b0d56 None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:30.753641 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a166dd-09a3-4316-9467-585eb39b0d56 None admin] RBAC: Adding query filter params (name=member) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:30.754151 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a166dd-09a3-4316-9467-585eb39b0d56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:30.755402 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a166dd-09a3-4316-9467-585eb39b0d56 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:30.763927 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 147/301] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 17 23:30:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:30.765377 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b26b1-e71c-4bb1-900f-f1e11ac47fd1 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:30.766467 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b26b1-e71c-4bb1-900f-f1e11ac47fd1 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:30.766672 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b26b1-e71c-4bb1-900f-f1e11ac47fd1 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:30.768061 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 148/302] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:30.772556 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b26b1-e71c-4bb1-900f-f1e11ac47fd1 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:30.775542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 155/303] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:30.781391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef86d7de-cccb-4a47-8e7a-6dc99a7df56f None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:30.781952 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef86d7de-cccb-4a47-8e7a-6dc99a7df56f None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:30.782957 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef86d7de-cccb-4a47-8e7a-6dc99a7df56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:30.808072 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-ef86d7de-cccb-4a47-8e7a-6dc99a7df56f None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:30.809966 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-ef86d7de-cccb-4a47-8e7a-6dc99a7df56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:30.880165 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:30.891803 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b26b1-e71c-4bb1-900f-f1e11ac47fd1 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:30.916662 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 156/304] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:30.924722 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef86d7de-cccb-4a47-8e7a-6dc99a7df56f None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:30.931371 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23632d1c-89f2-46e2-b4c1-05286d5131d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:30.932334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23632d1c-89f2-46e2-b4c1-05286d5131d2 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:30.932818 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23632d1c-89f2-46e2-b4c1-05286d5131d2 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:30.933804 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23632d1c-89f2-46e2-b4c1-05286d5131d2 None admin] PATH_INFO: `/v3/users/0bbf609b8b15441d8ee7b89898880156` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:30.935681 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23632d1c-89f2-46e2-b4c1-05286d5131d2 None admin] RBAC: Authorizing `identity:get_user(user_id=0bbf609b8b15441d8ee7b89898880156)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:30.938058 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23632d1c-89f2-46e2-b4c1-05286d5131d2 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:30.940865 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 149/305] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:30:30 2022] GET /identity/v3/users/0bbf609b8b15441d8ee7b89898880156 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:30.948013 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23632d1c-89f2-46e2-b4c1-05286d5131d2 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:30.961943 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:31.033453 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef86d7de-cccb-4a47-8e7a-6dc99a7df56f None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:31.062426 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23632d1c-89f2-46e2-b4c1-05286d5131d2 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:31.067736 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c20b6-2d25-482f-b5a7-9893c3a0220b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:31.068392 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c20b6-2d25-482f-b5a7-9893c3a0220b None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:31.068668 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c20b6-2d25-482f-b5a7-9893c3a0220b None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:31.068937 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c20b6-2d25-482f-b5a7-9893c3a0220b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:31.080109 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8c20b6-2d25-482f-b5a7-9893c3a0220b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:31.084986 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-6e8c20b6-2d25-482f-b5a7-9893c3a0220b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 23:30:31.084986 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:31.084986 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:31.084986 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:31.084986 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:31.084986 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:31.084986 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:31.084986 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:31.084986 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:31.084986 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:31.084986 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:31.084986 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:31.084986 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:31.084986 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:31.084986 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:31.084986 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:31.086708 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:31.086708 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:31.086708 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:31.086708 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:31.086708 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:31.086708 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:31.086708 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:31.086708 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:31.086708 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:31.086708 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:31.086708 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:31.086708 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:31.086708 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:31.086708 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:31.086708 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:31.086708 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:31.087731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:31.087731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:31.087731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:31.087731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:31.087731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:31.087731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:31.087731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:31.087731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:31.087731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:31.087731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:31.087731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 23:30:31.087731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:30:31.087731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 157/306] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:30:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:30:31.087731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c20b6-2d25-482f-b5a7-9893c3a0220b None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:31.106826 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba83f8-b593-4d83-aafa-7b5c25f5c0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:31.107481 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba83f8-b593-4d83-aafa-7b5c25f5c0df None admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:31.107757 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba83f8-b593-4d83-aafa-7b5c25f5c0df None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:31.108059 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba83f8-b593-4d83-aafa-7b5c25f5c0df None admin] PATH_INFO: `/v3/system/users/0bbf609b8b15441d8ee7b89898880156/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:31.109465 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba83f8-b593-4d83-aafa-7b5c25f5c0df None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0bbf609b8b15441d8ee7b89898880156, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:31.110842 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba83f8-b593-4d83-aafa-7b5c25f5c0df None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:31.120965 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 150/307] 10.222.0.26 () {58 vars in 1334 bytes} [Wed Aug 17 23:30:30 2022] PUT /identity/v3/system/users/0bbf609b8b15441d8ee7b89898880156/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:30:31.122037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba83f8-b593-4d83-aafa-7b5c25f5c0df None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:31.159536 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:31.230001 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:31.238162 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c20b6-2d25-482f-b5a7-9893c3a0220b None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:31.246638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba83f8-b593-4d83-aafa-7b5c25f5c0df None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:31.278621 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf176643-9455-42a9-aa35-ff8d9f008389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:31.279261 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf176643-9455-42a9-aa35-ff8d9f008389 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:31.279536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf176643-9455-42a9-aa35-ff8d9f008389 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:31.279817 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf176643-9455-42a9-aa35-ff8d9f008389 None admin] PATH_INFO: `/v3/domains/default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:31.281121 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf176643-9455-42a9-aa35-ff8d9f008389 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:31.282595 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf176643-9455-42a9-aa35-ff8d9f008389 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:31.285331 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 158/308] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:30:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:31.285331 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf176643-9455-42a9-aa35-ff8d9f008389 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:31.292908 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90deba24-9bef-49e3-a7ea-8dc63a462d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:31.293581 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90deba24-9bef-49e3-a7ea-8dc63a462d87 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:31.293862 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90deba24-9bef-49e3-a7ea-8dc63a462d87 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:31.294136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90deba24-9bef-49e3-a7ea-8dc63a462d87 None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:31.295242 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90deba24-9bef-49e3-a7ea-8dc63a462d87 None admin] RBAC: Adding query filter params (name=reader) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:31.295547 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90deba24-9bef-49e3-a7ea-8dc63a462d87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:31.297361 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90deba24-9bef-49e3-a7ea-8dc63a462d87 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:31.310734 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 151/309] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 17 23:30:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:31.310734 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90deba24-9bef-49e3-a7ea-8dc63a462d87 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:31.401680 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf176643-9455-42a9-aa35-ff8d9f008389 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:31.408636 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:31.419662 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90deba24-9bef-49e3-a7ea-8dc63a462d87 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:31.448616 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b532f7-471e-4a9e-8135-c9e5c29cf3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:31.449309 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b532f7-471e-4a9e-8135-c9e5c29cf3ff None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:31.449593 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b532f7-471e-4a9e-8135-c9e5c29cf3ff None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:31.450440 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b532f7-471e-4a9e-8135-c9e5c29cf3ff None admin] PATH_INFO: `/v3/roles/reader` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:31.458637 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d1f82-f80e-4e1c-93a4-25029e18fb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:31.459313 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d1f82-f80e-4e1c-93a4-25029e18fb2b None admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:31.459664 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d1f82-f80e-4e1c-93a4-25029e18fb2b None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:31.460034 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d1f82-f80e-4e1c-93a4-25029e18fb2b None admin] PATH_INFO: `/v3/groups` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:31.460749 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d1f82-f80e-4e1c-93a4-25029e18fb2b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:31.461463 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b532f7-471e-4a9e-8135-c9e5c29cf3ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:31.462325 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d1f82-f80e-4e1c-93a4-25029e18fb2b None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:31.467309 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-d7b532f7-471e-4a9e-8135-c9e5c29cf3ff None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 23:30:31.467309 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:31.467309 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:31.467309 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:31.467309 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:31.467309 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:31.467309 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:31.467309 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:31.467309 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:31.467309 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:31.467309 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:31.467309 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:31.467309 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:31.467309 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:31.467309 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:31.467309 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:31.468252 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:31.468252 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:31.468252 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:31.468252 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:31.468252 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:31.468252 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:31.468252 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:31.468252 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:31.468252 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:31.468252 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:31.468252 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:31.468252 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:31.468252 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:31.468252 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:31.468252 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:31.468252 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:31.469048 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:31.469048 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:31.469048 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:31.469048 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:31.469048 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:31.469048 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:31.469048 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:31.469048 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:31.469048 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:31.469048 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:31.469048 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 23:30:31.469048 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:30:31.469048 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 159/310] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:30:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:31.469701 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b532f7-471e-4a9e-8135-c9e5c29cf3ff None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:31.483731 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 152/311] 10.222.0.26 () {60 vars in 1134 bytes} [Wed Aug 17 23:30:31 2022] POST /identity/v3/groups => generated 229 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:30:31.484422 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d1f82-f80e-4e1c-93a4-25029e18fb2b None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:31.523393 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:31.625904 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d1f82-f80e-4e1c-93a4-25029e18fb2b None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:31.632685 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b532f7-471e-4a9e-8135-c9e5c29cf3ff None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:31.671857 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fcad1e-ddfa-4a42-9365-33256959719a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:31.673061 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fcad1e-ddfa-4a42-9365-33256959719a None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:31.673951 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fcad1e-ddfa-4a42-9365-33256959719a None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:31.674846 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fcad1e-ddfa-4a42-9365-33256959719a None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:31.675666 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fcad1e-ddfa-4a42-9365-33256959719a None admin] RBAC: Adding query filter params (name=reader) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:31.676161 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fcad1e-ddfa-4a42-9365-33256959719a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:31.677686 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fcad1e-ddfa-4a42-9365-33256959719a None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:31.677947 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55f6ad-1eb9-4612-b3aa-4e55d2b8fa9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:31.678580 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55f6ad-1eb9-4612-b3aa-4e55d2b8fa9e None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:31.678867 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55f6ad-1eb9-4612-b3aa-4e55d2b8fa9e None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:31.679139 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55f6ad-1eb9-4612-b3aa-4e55d2b8fa9e None admin] PATH_INFO: `/v3/users/760c062bf340436bb383e31b550c91e5` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:31.689820 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 153/312] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 17 23:30:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:31.695460 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fcad1e-ddfa-4a42-9365-33256959719a None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:31.700199 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca55f6ad-1eb9-4612-b3aa-4e55d2b8fa9e None admin] RBAC: Authorizing `identity:get_user(user_id=760c062bf340436bb383e31b550c91e5)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:31.702412 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca55f6ad-1eb9-4612-b3aa-4e55d2b8fa9e None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:31.704336 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 160/313] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:30:31 2022] GET /identity/v3/users/760c062bf340436bb383e31b550c91e5 => generated 293 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:31.710318 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55f6ad-1eb9-4612-b3aa-4e55d2b8fa9e None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:31.716182 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:31.816435 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:31.820807 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55f6ad-1eb9-4612-b3aa-4e55d2b8fa9e None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:31.832947 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fcad1e-ddfa-4a42-9365-33256959719a None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:31.878349 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c30551-a090-4723-85f9-e6c7a4db3d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:31.879355 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62917819-5bd5-45e4-8144-cc5e4bac1c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:31.879658 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c30551-a090-4723-85f9-e6c7a4db3d33 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:31.880071 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62917819-5bd5-45e4-8144-cc5e4bac1c95 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:31.880354 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62917819-5bd5-45e4-8144-cc5e4bac1c95 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:31.880613 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c30551-a090-4723-85f9-e6c7a4db3d33 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:31.880712 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62917819-5bd5-45e4-8144-cc5e4bac1c95 None admin] PATH_INFO: `/v3/users/fbac01ecde4f45afa6619ea7c526f440` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:31.881303 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c30551-a090-4723-85f9-e6c7a4db3d33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:31.881750 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62917819-5bd5-45e4-8144-cc5e4bac1c95 None admin] RBAC: Authorizing `identity:get_user(user_id=fbac01ecde4f45afa6619ea7c526f440)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:31.882439 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c30551-a090-4723-85f9-e6c7a4db3d33 None admin] RBAC: Adding query filter params (role.id=df3442912b3d43f0a36ebb936b3cb102, user.id=760c062bf340436bb383e31b550c91e5, scope.system=all) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:31.882965 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c30551-a090-4723-85f9-e6c7a4db3d33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:31.883203 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62917819-5bd5-45e4-8144-cc5e4bac1c95 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:31.884898 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c30551-a090-4723-85f9-e6c7a4db3d33 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:31.885253 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 154/314] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:30:31 2022] GET /identity/v3/users/fbac01ecde4f45afa6619ea7c526f440 => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:30:31.890865 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62917819-5bd5-45e4-8144-cc5e4bac1c95 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:31.891784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 161/315] 10.222.0.26 () {56 vars in 1308 bytes} [Wed Aug 17 23:30:31 2022] GET /identity/v3/role_assignments?user.id=760c062bf340436bb383e31b550c91e5&scope.system=all&role.id=df3442912b3d43f0a36ebb936b3cb102 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:31.987115 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:32.010966 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62917819-5bd5-45e4-8144-cc5e4bac1c95 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:32.048405 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6885ba9c-0727-4e25-9802-3be50e0701a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:32.049023 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6885ba9c-0727-4e25-9802-3be50e0701a2 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:32.049297 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6885ba9c-0727-4e25-9802-3be50e0701a2 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:32.049568 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6885ba9c-0727-4e25-9802-3be50e0701a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:32.125792 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6885ba9c-0727-4e25-9802-3be50e0701a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:32.127277 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6885ba9c-0727-4e25-9802-3be50e0701a2 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:32.215405 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:32.236289 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 155/316] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:30:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:30:32.242039 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c30551-a090-4723-85f9-e6c7a4db3d33 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:32.348713 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c30551-a090-4723-85f9-e6c7a4db3d33 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:32.385804 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637cf96c-effc-4c22-94a8-2d6fe5d32844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:32.386648 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637cf96c-effc-4c22-94a8-2d6fe5d32844 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:32.387153 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637cf96c-effc-4c22-94a8-2d6fe5d32844 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:32.388076 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637cf96c-effc-4c22-94a8-2d6fe5d32844 None admin] PATH_INFO: `/v3/projects/e959a87f43754d66b9bd795c08d2d827` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:32.389415 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637cf96c-effc-4c22-94a8-2d6fe5d32844 None admin] RBAC: Authorizing `identity:get_project(project_id=e959a87f43754d66b9bd795c08d2d827)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:32.391010 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637cf96c-effc-4c22-94a8-2d6fe5d32844 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:32.393095 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 162/317] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:32 2022] GET /identity/v3/projects/e959a87f43754d66b9bd795c08d2d827 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:32.399046 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6885ba9c-0727-4e25-9802-3be50e0701a2 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:32.505705 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6885ba9c-0727-4e25-9802-3be50e0701a2 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:32.541313 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac2fc2-002b-4151-9ec1-5390388ada14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:32.542257 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac2fc2-002b-4151-9ec1-5390388ada14 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:32.542732 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac2fc2-002b-4151-9ec1-5390388ada14 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:32.543203 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac2fc2-002b-4151-9ec1-5390388ada14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:32.544687 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ac2fc2-002b-4151-9ec1-5390388ada14 None admin] RBAC: Adding query filter params (role.id=df3442912b3d43f0a36ebb936b3cb102, scope.project.id=e959a87f43754d66b9bd795c08d2d827, user.id=fbac01ecde4f45afa6619ea7c526f440) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:32.545239 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ac2fc2-002b-4151-9ec1-5390388ada14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:32.546896 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ac2fc2-002b-4151-9ec1-5390388ada14 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:32.554095 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 156/318] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:30:32 2022] GET /identity/v3/role_assignments?user.id=fbac01ecde4f45afa6619ea7c526f440&scope.project.id=e959a87f43754d66b9bd795c08d2d827&role.id=df3442912b3d43f0a36ebb936b3cb102 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:33.032703 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db115206-65c7-419a-992b-ec5b9095a408 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:33.033080 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db115206-65c7-419a-992b-ec5b9095a408 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:33.033281 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db115206-65c7-419a-992b-ec5b9095a408 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:33.034621 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 163/319] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:33.040783 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd966f9-b2bd-41b5-ab4f-0bc987ba8d2b None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:33.041278 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd966f9-b2bd-41b5-ab4f-0bc987ba8d2b None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:33.041888 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd966f9-b2bd-41b5-ab4f-0bc987ba8d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:33.061245 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-9dd966f9-b2bd-41b5-ab4f-0bc987ba8d2b None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:33.062429 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-9dd966f9-b2bd-41b5-ab4f-0bc987ba8d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:33.103722 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:33.131482 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 157/320] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:33.398493 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db115206-65c7-419a-992b-ec5b9095a408 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:33.415052 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:33.502164 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcba2f6-2049-4aec-a9bf-04b2c198ec3b None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:33.502479 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcba2f6-2049-4aec-a9bf-04b2c198ec3b None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:33.502714 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcba2f6-2049-4aec-a9bf-04b2c198ec3b None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:33.504192 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 158/321] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 23:30:33.508803 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5449b-728d-4410-b4f3-81144d8065f5 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:33.509174 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5449b-728d-4410-b4f3-81144d8065f5 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:33.509445 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5449b-728d-4410-b4f3-81144d8065f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:33.510267 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db115206-65c7-419a-992b-ec5b9095a408 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:33.526563 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-24d5449b-728d-4410-b4f3-81144d8065f5 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:33.528068 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-24d5449b-728d-4410-b4f3-81144d8065f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:33.548288 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efa6419-3d6d-4501-ae95-3b9aff7f5c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:33.549219 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa6419-3d6d-4501-ae95-3b9aff7f5c65 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:33.549507 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa6419-3d6d-4501-ae95-3b9aff7f5c65 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:33.549808 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa6419-3d6d-4501-ae95-3b9aff7f5c65 None admin] PATH_INFO: `/v3/roles/member` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:33.562746 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efa6419-3d6d-4501-ae95-3b9aff7f5c65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:33.567536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-4efa6419-3d6d-4501-ae95-3b9aff7f5c65 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 23:30:33.567536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:33.567536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:33.567536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:33.567536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:33.567536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:33.567536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:33.567536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:33.567536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:33.567536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:33.567536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:33.567536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:33.567536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:33.567536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:33.567536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:33.567536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:33.568518 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:33.568518 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:33.568518 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:33.568518 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:33.568518 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:33.568518 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:33.568518 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:33.568518 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:33.568518 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:33.568518 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:33.568518 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:33.568518 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:33.568518 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:33.568518 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:33.568518 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:33.568518 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:33.569471 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:33.569471 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:33.569471 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:33.569471 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:33.569471 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:33.569471 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:33.569471 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:33.569471 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:33.569471 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:33.569471 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:33.569471 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 23:30:33.569471 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:30:33.569471 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 164/322] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:30:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:33.574135 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efa6419-3d6d-4501-ae95-3b9aff7f5c65 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:33.605057 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 159/323] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 23:30:33.653964 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:33.680598 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efa6419-3d6d-4501-ae95-3b9aff7f5c65 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:33.716581 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc80ac5-493c-43e0-b90a-9b3029cf2e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:33.717187 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc80ac5-493c-43e0-b90a-9b3029cf2e80 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:33.717566 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc80ac5-493c-43e0-b90a-9b3029cf2e80 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:33.717837 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc80ac5-493c-43e0-b90a-9b3029cf2e80 None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:33.718502 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc80ac5-493c-43e0-b90a-9b3029cf2e80 None admin] RBAC: Adding query filter params (name=member) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:33.718803 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc80ac5-493c-43e0-b90a-9b3029cf2e80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:33.720204 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc80ac5-493c-43e0-b90a-9b3029cf2e80 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:33.733477 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 165/324] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 17 23:30:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:33.734872 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d5449b-728d-4410-b4f3-81144d8065f5 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:33.836658 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d5449b-728d-4410-b4f3-81144d8065f5 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:33.844339 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:33.865137 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5cbaa-e98b-4267-8c26-68535aa793b5 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:33.865420 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5cbaa-e98b-4267-8c26-68535aa793b5 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:33.865693 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5cbaa-e98b-4267-8c26-68535aa793b5 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:33.867033 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 166/325] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 23:30:33.871903 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354aef93-7ed0-450c-b9d7-28f4c1f0f440 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:33.872230 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354aef93-7ed0-450c-b9d7-28f4c1f0f440 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:33.872508 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354aef93-7ed0-450c-b9d7-28f4c1f0f440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:33.878887 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf748b-4af0-4750-b010-49bdc015683d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:33.879519 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf748b-4af0-4750-b010-49bdc015683d None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:33.879795 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf748b-4af0-4750-b010-49bdc015683d None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:33.880121 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf748b-4af0-4750-b010-49bdc015683d None admin] PATH_INFO: `/v3/users/0bbf609b8b15441d8ee7b89898880156` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:33.881236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf748b-4af0-4750-b010-49bdc015683d None admin] RBAC: Authorizing `identity:get_user(user_id=0bbf609b8b15441d8ee7b89898880156)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:33.882655 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf748b-4af0-4750-b010-49bdc015683d None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:33.884250 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 160/326] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:30:33 2022] GET /identity/v3/users/0bbf609b8b15441d8ee7b89898880156 => generated 293 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:33.886192 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf748b-4af0-4750-b010-49bdc015683d None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:33.890515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-354aef93-7ed0-450c-b9d7-28f4c1f0f440 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:33.891719 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-354aef93-7ed0-450c-b9d7-28f4c1f0f440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:33.957755 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 167/327] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 23:30:33.958573 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354aef93-7ed0-450c-b9d7-28f4c1f0f440 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:33.990431 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf748b-4af0-4750-b010-49bdc015683d None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:34.027204 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:34.027352 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e343176-8e43-403c-a0bd-56683b7a6397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:34.028751 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e343176-8e43-403c-a0bd-56683b7a6397 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:34.029036 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e343176-8e43-403c-a0bd-56683b7a6397 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:34.029305 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e343176-8e43-403c-a0bd-56683b7a6397 None admin] PATH_INFO: `/v3/domains/default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:34.030797 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e343176-8e43-403c-a0bd-56683b7a6397 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:34.032398 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e343176-8e43-403c-a0bd-56683b7a6397 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:34.034140 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 161/328] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:30:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:34.039344 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e343176-8e43-403c-a0bd-56683b7a6397 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:34.067023 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354aef93-7ed0-450c-b9d7-28f4c1f0f440 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:34.086810 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:34.111124 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6b818-5a6a-4b76-a9a2-eda50611d459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:34.111734 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6b818-5a6a-4b76-a9a2-eda50611d459 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:34.112036 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6b818-5a6a-4b76-a9a2-eda50611d459 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:34.112308 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6b818-5a6a-4b76-a9a2-eda50611d459 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:34.113029 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6b818-5a6a-4b76-a9a2-eda50611d459 None admin] RBAC: Adding query filter params (role.id=1182317a465349478dde4e28cb7cf96f, user.id=0bbf609b8b15441d8ee7b89898880156, scope.system=all) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:34.113335 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6b818-5a6a-4b76-a9a2-eda50611d459 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:34.114776 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6b818-5a6a-4b76-a9a2-eda50611d459 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:34.126457 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 168/329] 10.222.0.26 () {56 vars in 1308 bytes} [Wed Aug 17 23:30:33 2022] GET /identity/v3/role_assignments?user.id=0bbf609b8b15441d8ee7b89898880156&scope.system=all&role.id=1182317a465349478dde4e28cb7cf96f => generated 510 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:34.153398 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e343176-8e43-403c-a0bd-56683b7a6397 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:34.194084 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de990f-7e01-4a3f-bfd4-debbcc7a41d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:34.194706 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de990f-7e01-4a3f-bfd4-debbcc7a41d1 None admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:34.194989 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de990f-7e01-4a3f-bfd4-debbcc7a41d1 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:34.195260 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de990f-7e01-4a3f-bfd4-debbcc7a41d1 None admin] PATH_INFO: `/v3/groups` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:34.195898 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de990f-7e01-4a3f-bfd4-debbcc7a41d1 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:34.197442 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de990f-7e01-4a3f-bfd4-debbcc7a41d1 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:34.216770 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 162/330] 10.222.0.26 () {60 vars in 1134 bytes} [Wed Aug 17 23:30:34 2022] POST /identity/v3/groups => generated 226 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:30:34.249302 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6b818-5a6a-4b76-a9a2-eda50611d459 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:34.367345 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6b818-5a6a-4b76-a9a2-eda50611d459 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:34.391135 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:34.405157 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc52bcb-e5ef-4de2-8b86-086885bdb223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:34.405780 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc52bcb-e5ef-4de2-8b86-086885bdb223 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:34.406054 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc52bcb-e5ef-4de2-8b86-086885bdb223 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:34.406320 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc52bcb-e5ef-4de2-8b86-086885bdb223 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:34.416930 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc52bcb-e5ef-4de2-8b86-086885bdb223 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:34.422167 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-bfc52bcb-e5ef-4de2-8b86-086885bdb223 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 23:30:34.422167 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:34.422167 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:34.422167 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:34.422167 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:34.422167 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:34.422167 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:34.422167 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:34.422167 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:34.422167 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:34.422167 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:34.422167 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:34.422167 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:34.422167 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:34.422167 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:34.422167 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:34.423072 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:34.423072 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:34.423072 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:34.423072 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:34.423072 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:34.423072 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:34.423072 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:34.423072 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:34.423072 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:34.423072 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:34.423072 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:34.423072 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:34.423072 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:34.423072 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:34.423072 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:34.423072 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:34.423899 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:34.423899 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:34.423899 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:34.423899 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:34.423899 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:34.423899 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:34.423899 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:34.423899 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:34.423899 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:34.423899 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:34.423899 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 23:30:34.423899 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:30:34.423899 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 169/331] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:30:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:34.429176 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de990f-7e01-4a3f-bfd4-debbcc7a41d1 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:34.542991 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de990f-7e01-4a3f-bfd4-debbcc7a41d1 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:34.544615 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:34.581731 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7a4e1a-80e9-4e06-bffe-fb8d9f0f62fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:34.582348 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a4e1a-80e9-4e06-bffe-fb8d9f0f62fb None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:34.582627 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a4e1a-80e9-4e06-bffe-fb8d9f0f62fb None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:34.582898 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a4e1a-80e9-4e06-bffe-fb8d9f0f62fb None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:34.583596 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7a4e1a-80e9-4e06-bffe-fb8d9f0f62fb None admin] RBAC: Adding query filter params (name=reader) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:34.583869 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7a4e1a-80e9-4e06-bffe-fb8d9f0f62fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:34.585453 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7a4e1a-80e9-4e06-bffe-fb8d9f0f62fb None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:34.595148 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 163/332] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 17 23:30:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:34.600543 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc52bcb-e5ef-4de2-8b86-086885bdb223 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:34.717809 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc52bcb-e5ef-4de2-8b86-086885bdb223 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:34.756357 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb35cf-b774-4fad-8d39-5194181ed418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:34.757045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb35cf-b774-4fad-8d39-5194181ed418 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:34.757327 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb35cf-b774-4fad-8d39-5194181ed418 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:34.757600 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb35cf-b774-4fad-8d39-5194181ed418 None admin] PATH_INFO: `/v3/users/760c062bf340436bb383e31b550c91e5` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:34.758838 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cb35cf-b774-4fad-8d39-5194181ed418 None admin] RBAC: Authorizing `identity:get_user(user_id=760c062bf340436bb383e31b550c91e5)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:34.760340 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cb35cf-b774-4fad-8d39-5194181ed418 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:34.762831 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 170/333] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:30:34 2022] GET /identity/v3/users/760c062bf340436bb383e31b550c91e5 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:34.768318 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7a4e1a-80e9-4e06-bffe-fb8d9f0f62fb None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:34.886352 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7a4e1a-80e9-4e06-bffe-fb8d9f0f62fb None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:34.926080 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b61e7b8-009d-4245-9b57-84e970da677f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:34.926735 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61e7b8-009d-4245-9b57-84e970da677f None admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:34.927015 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61e7b8-009d-4245-9b57-84e970da677f None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:34.927291 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61e7b8-009d-4245-9b57-84e970da677f None admin] PATH_INFO: `/v3/system/users/760c062bf340436bb383e31b550c91e5/roles/df3442912b3d43f0a36ebb936b3cb102` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:34.928762 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b61e7b8-009d-4245-9b57-84e970da677f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=760c062bf340436bb383e31b550c91e5, role_id=df3442912b3d43f0a36ebb936b3cb102)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:34.930148 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b61e7b8-009d-4245-9b57-84e970da677f None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:34.939725 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 164/334] 10.222.0.26 () {58 vars in 1334 bytes} [Wed Aug 17 23:30:34 2022] PUT /identity/v3/system/users/760c062bf340436bb383e31b550c91e5/roles/df3442912b3d43f0a36ebb936b3cb102 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:30:36.587276 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc1bd5-c43d-4717-a139-0e9a160e55cd None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:36.587803 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc1bd5-c43d-4717-a139-0e9a160e55cd None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:36.587803 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc1bd5-c43d-4717-a139-0e9a160e55cd None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:36.589636 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 171/335] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:36.594895 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49fa693-6ef0-410b-bce5-1d290d43899f None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:36.595367 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49fa693-6ef0-410b-bce5-1d290d43899f None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:36.595453 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49fa693-6ef0-410b-bce5-1d290d43899f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:36.626963 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-c49fa693-6ef0-410b-bce5-1d290d43899f None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:36.628325 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-c49fa693-6ef0-410b-bce5-1d290d43899f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:36.668961 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:36.758139 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 165/336] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:36.991579 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8d20a-f8bd-44f4-8530-e84be1f2d324 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:36.991889 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8d20a-f8bd-44f4-8530-e84be1f2d324 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:36.992167 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8d20a-f8bd-44f4-8530-e84be1f2d324 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:36.993681 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 172/337] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:36.999137 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043b52c-d2ee-40b9-a9ea-cde74c198737 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:36.999429 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043b52c-d2ee-40b9-a9ea-cde74c198737 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:36.999708 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043b52c-d2ee-40b9-a9ea-cde74c198737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:37.022437 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-f043b52c-d2ee-40b9-a9ea-cde74c198737 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:37.023548 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-f043b52c-d2ee-40b9-a9ea-cde74c198737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:37.145607 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 166/338] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:37.285068 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e8d20a-f8bd-44f4-8530-e84be1f2d324 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:37.352617 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:37.419762 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e8d20a-f8bd-44f4-8530-e84be1f2d324 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:37.480060 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb078db-2dcf-4295-b3a9-e459c85b0ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:37.480741 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb078db-2dcf-4295-b3a9-e459c85b0ea2 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:37.481023 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb078db-2dcf-4295-b3a9-e459c85b0ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:37.481296 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb078db-2dcf-4295-b3a9-e459c85b0ea2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:37.500548 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb078db-2dcf-4295-b3a9-e459c85b0ea2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:37.505872 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-dfb078db-2dcf-4295-b3a9-e459c85b0ea2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 23:30:37.505872 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:37.505872 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:37.505872 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:37.505872 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:37.505872 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:37.505872 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:37.505872 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:37.505872 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:37.505872 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:37.505872 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:37.505872 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:37.505872 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:37.505872 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:37.505872 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:37.505872 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:37.506744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:37.506744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:37.506744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:37.506744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:37.506744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:37.506744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:37.506744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:37.506744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:37.506744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:37.506744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:37.506744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:37.506744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:37.506744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:37.506744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:37.506744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:37.506744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:37.507547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:37.507547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:37.507547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:37.507547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:37.507547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:37.507547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:37.507547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:37.507547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:37.507547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:37.507547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:37.507547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 23:30:37.507547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:30:37.507547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 173/339] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:30:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:37.514085 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043b52c-d2ee-40b9-a9ea-cde74c198737 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:37.564109 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd0505-c95e-4279-a328-3f79572d64b2 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:37.564272 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd0505-c95e-4279-a328-3f79572d64b2 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:37.564497 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd0505-c95e-4279-a328-3f79572d64b2 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:37.566142 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 174/340] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:37.578455 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e677e5-5ba5-41e0-a097-320f24204ae2 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:37.578939 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e677e5-5ba5-41e0-a097-320f24204ae2 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:37.579338 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e677e5-5ba5-41e0-a097-320f24204ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:37.613434 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-32e677e5-5ba5-41e0-a097-320f24204ae2 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:37.615554 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-32e677e5-5ba5-41e0-a097-320f24204ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:37.736208 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 175/341] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:37.737361 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e677e5-5ba5-41e0-a097-320f24204ae2 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:37.768028 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:37.790887 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043b52c-d2ee-40b9-a9ea-cde74c198737 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:37.821898 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:37.871900 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1ef97-2a42-4ab7-a8ef-ea0fc15dfbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:37.872803 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1ef97-2a42-4ab7-a8ef-ea0fc15dfbe5 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:37.873225 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1ef97-2a42-4ab7-a8ef-ea0fc15dfbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:37.873644 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1ef97-2a42-4ab7-a8ef-ea0fc15dfbe5 None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:37.874492 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc1ef97-2a42-4ab7-a8ef-ea0fc15dfbe5 None admin] RBAC: Adding query filter params (name=member) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:37.874937 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc1ef97-2a42-4ab7-a8ef-ea0fc15dfbe5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:37.876490 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc1ef97-2a42-4ab7-a8ef-ea0fc15dfbe5 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:37.896471 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 167/342] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 17 23:30:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:37.898844 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1ef97-2a42-4ab7-a8ef-ea0fc15dfbe5 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:37.900837 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e677e5-5ba5-41e0-a097-320f24204ae2 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:37.961070 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780221e5-d31a-447c-8de9-5486951216d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:37.961724 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780221e5-d31a-447c-8de9-5486951216d7 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:37.962001 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780221e5-d31a-447c-8de9-5486951216d7 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:37.962268 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780221e5-d31a-447c-8de9-5486951216d7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:37.986701 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780221e5-d31a-447c-8de9-5486951216d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:37.991445 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-780221e5-d31a-447c-8de9-5486951216d7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 23:30:37.991445 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:37.991445 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:37.991445 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:37.991445 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:37.991445 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:37.991445 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:37.991445 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:37.991445 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:37.991445 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:37.991445 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:37.991445 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:37.991445 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:37.991445 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:37.991445 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:37.991445 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:37.992515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:37.992515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:37.992515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:37.992515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:37.992515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:37.992515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:37.992515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:37.992515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:37.992515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:37.992515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:37.992515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:37.992515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:37.992515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:37.992515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:37.992515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:37.992515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:37.993320 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:37.993320 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:37.993320 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:37.993320 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:37.993320 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:37.993320 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:37.993320 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:37.993320 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:37.993320 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:37.993320 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:37.993320 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 23:30:37.993320 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:30:38.000259 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 176/343] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 17 23:30:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:38.009745 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780221e5-d31a-447c-8de9-5486951216d7 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:38.114372 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1ef97-2a42-4ab7-a8ef-ea0fc15dfbe5 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:38.168195 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0acc2b-4ee4-45e4-9781-88e1d8d0e423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:38.170803 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0acc2b-4ee4-45e4-9781-88e1d8d0e423 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:38.173217 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0acc2b-4ee4-45e4-9781-88e1d8d0e423 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:38.173774 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0acc2b-4ee4-45e4-9781-88e1d8d0e423 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:38.197844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780221e5-d31a-447c-8de9-5486951216d7 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:38.200583 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:38.221534 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:38.284818 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1b41f2-d5ef-40f6-99db-67e97ae1b784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:38.285453 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b41f2-d5ef-40f6-99db-67e97ae1b784 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:38.285833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b41f2-d5ef-40f6-99db-67e97ae1b784 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:38.286026 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b41f2-d5ef-40f6-99db-67e97ae1b784 None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:38.286688 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1b41f2-d5ef-40f6-99db-67e97ae1b784 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:38.286998 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1b41f2-d5ef-40f6-99db-67e97ae1b784 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:38.287564 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0acc2b-4ee4-45e4-9781-88e1d8d0e423 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:38.288449 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1b41f2-d5ef-40f6-99db-67e97ae1b784 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:38.289528 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0acc2b-4ee4-45e4-9781-88e1d8d0e423 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:38.300116 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 177/344] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:30:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:38.300116 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7524f9-fdd2-48a5-b0c8-45beca75a48c None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:38.300116 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7524f9-fdd2-48a5-b0c8-45beca75a48c None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:38.300116 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7524f9-fdd2-48a5-b0c8-45beca75a48c None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:38.308659 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 178/345] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:38.308659 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7524f9-fdd2-48a5-b0c8-45beca75a48c None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:38.408166 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 168/346] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:30:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:30:38.409262 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0acc2b-4ee4-45e4-9781-88e1d8d0e423 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:38.485335 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:38.502258 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7524f9-fdd2-48a5-b0c8-45beca75a48c None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:38.556731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e9330b-857e-4612-a5a3-dabe76232ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:38.557662 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9330b-857e-4612-a5a3-dabe76232ff6 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:38.558115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9330b-857e-4612-a5a3-dabe76232ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:38.558554 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9330b-857e-4612-a5a3-dabe76232ff6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:38.560908 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0acc2b-4ee4-45e4-9781-88e1d8d0e423 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:38.573027 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e9330b-857e-4612-a5a3-dabe76232ff6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:38.579201 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-05e9330b-857e-4612-a5a3-dabe76232ff6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 23:30:38.579201 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:38.579201 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:38.579201 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:38.579201 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:38.579201 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:38.579201 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:38.579201 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:38.579201 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:38.579201 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:38.579201 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:38.579201 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:38.579201 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:38.579201 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:38.579201 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:38.579201 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:38.580213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:38.580213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:38.580213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:38.580213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:38.580213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:38.580213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:38.580213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:38.580213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:38.580213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:38.580213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:38.580213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:38.580213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:38.580213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:38.580213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:38.580213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:38.580213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:38.581099 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:38.581099 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:38.581099 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:38.581099 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:38.581099 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:38.581099 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:38.581099 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:38.581099 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:38.581099 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:38.581099 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:38.581099 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 23:30:38.581099 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:30:38.583052 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 179/347] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:30:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:38.584386 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ff83e-8acd-45f7-8536-d76e093eb410 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:38.584797 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ff83e-8acd-45f7-8536-d76e093eb410 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:38.585934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ff83e-8acd-45f7-8536-d76e093eb410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:38.612798 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-6b9ff83e-8acd-45f7-8536-d76e093eb410 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:38.617444 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-6b9ff83e-8acd-45f7-8536-d76e093eb410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:38.622574 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a11d442-e8a2-4c37-856c-471519310ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:38.623200 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a11d442-e8a2-4c37-856c-471519310ac5 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:38.623479 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a11d442-e8a2-4c37-856c-471519310ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:38.623747 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a11d442-e8a2-4c37-856c-471519310ac5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:38.644090 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:38.734770 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 180/348] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:38.735628 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ff83e-8acd-45f7-8536-d76e093eb410 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:38.736870 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a11d442-e8a2-4c37-856c-471519310ac5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:38.738531 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a11d442-e8a2-4c37-856c-471519310ac5 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:38.810584 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:38.882160 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ff83e-8acd-45f7-8536-d76e093eb410 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:38.894567 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 169/349] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:30:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:30:38.896259 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f8b613-7b92-4551-8865-648b46c5fc96 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:38.896748 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f8b613-7b92-4551-8865-648b46c5fc96 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:38.897262 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f8b613-7b92-4551-8865-648b46c5fc96 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:38.899117 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 170/350] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:38.901535 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f8b613-7b92-4551-8865-648b46c5fc96 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:38.930203 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e6313-4897-46e4-8817-9c7c7401eb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:38.930841 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e6313-4897-46e4-8817-9c7c7401eb4b None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:38.931117 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e6313-4897-46e4-8817-9c7c7401eb4b None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:38.931384 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e6313-4897-46e4-8817-9c7c7401eb4b None admin] PATH_INFO: `/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:38.934512 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e6313-4897-46e4-8817-9c7c7401eb4b None admin] RBAC: Authorizing `identity:get_project(project_id=105a2b33e5ee4cf88b11c48aebf329d7)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:38.936001 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e6313-4897-46e4-8817-9c7c7401eb4b None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:38.942384 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 181/351] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:38 2022] GET /identity/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:38.942384 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e6313-4897-46e4-8817-9c7c7401eb4b None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:38.974434 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:39.075808 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f8b613-7b92-4551-8865-648b46c5fc96 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:39.087737 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e6313-4897-46e4-8817-9c7c7401eb4b None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:39.133064 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18092a29-0f09-47d9-a356-1137ddf17819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:39.133810 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18092a29-0f09-47d9-a356-1137ddf17819 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:39.134097 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18092a29-0f09-47d9-a356-1137ddf17819 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:39.134374 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18092a29-0f09-47d9-a356-1137ddf17819 None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:39.135053 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18092a29-0f09-47d9-a356-1137ddf17819 None admin] RBAC: Adding query filter params (name=member) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:39.135350 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18092a29-0f09-47d9-a356-1137ddf17819 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:39.136809 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18092a29-0f09-47d9-a356-1137ddf17819 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:39.148445 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 171/352] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 17 23:30:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:39.149783 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01844e-1db5-48cc-8763-ca301124c579 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:39.150078 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01844e-1db5-48cc-8763-ca301124c579 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:39.150352 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01844e-1db5-48cc-8763-ca301124c579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:39.153584 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0fd71e-55b2-4ca5-9910-eb0bb415fbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:39.155394 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fd71e-55b2-4ca5-9910-eb0bb415fbbc None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:39.156866 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fd71e-55b2-4ca5-9910-eb0bb415fbbc None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:39.160176 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fd71e-55b2-4ca5-9910-eb0bb415fbbc None admin] PATH_INFO: `/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:39.162261 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0fd71e-55b2-4ca5-9910-eb0bb415fbbc None admin] RBAC: Authorizing `identity:get_project(project_id=105a2b33e5ee4cf88b11c48aebf329d7)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:39.163935 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0fd71e-55b2-4ca5-9910-eb0bb415fbbc None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:39.166145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 182/353] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:38 2022] GET /identity/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:39.167099 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0fd71e-55b2-4ca5-9910-eb0bb415fbbc None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:39.181070 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-2b01844e-1db5-48cc-8763-ca301124c579 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:39.182263 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-2b01844e-1db5-48cc-8763-ca301124c579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:39.194180 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:39.288185 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 172/354] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:39.289687 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01844e-1db5-48cc-8763-ca301124c579 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:39.322932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:39.338252 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0fd71e-55b2-4ca5-9910-eb0bb415fbbc None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:39.386117 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a47a6-7033-4844-a3be-1cab79136a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:39.387137 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a47a6-7033-4844-a3be-1cab79136a7c None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:39.387416 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a47a6-7033-4844-a3be-1cab79136a7c None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:39.387690 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a47a6-7033-4844-a3be-1cab79136a7c None admin] PATH_INFO: `/v3/groups/ac7c419a3715488f9ab314b2075879da` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:39.396209 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08a47a6-7033-4844-a3be-1cab79136a7c None admin] RBAC: Authorizing `identity:get_group(group_id=ac7c419a3715488f9ab314b2075879da)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:39.397675 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08a47a6-7033-4844-a3be-1cab79136a7c None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:39.405877 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 183/355] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Aug 17 23:30:39 2022] GET /identity/v3/groups/ac7c419a3715488f9ab314b2075879da => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:39.405877 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a47a6-7033-4844-a3be-1cab79136a7c None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:39.452052 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01844e-1db5-48cc-8763-ca301124c579 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:39.504357 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59461e8b-ddc9-4eb3-9ab4-18feba145474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:39.505247 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59461e8b-ddc9-4eb3-9ab4-18feba145474 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:39.506883 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59461e8b-ddc9-4eb3-9ab4-18feba145474 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:39.507305 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59461e8b-ddc9-4eb3-9ab4-18feba145474 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:39.527231 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59461e8b-ddc9-4eb3-9ab4-18feba145474 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:39.534701 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-59461e8b-ddc9-4eb3-9ab4-18feba145474 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 23:30:39.534701 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:39.534701 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:39.534701 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:39.534701 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:39.534701 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:39.534701 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:39.534701 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:39.534701 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:39.534701 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:39.534701 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:39.534701 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:39.534701 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:39.534701 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:39.534701 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:39.534701 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:39.535607 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:39.535607 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:39.535607 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:39.535607 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:39.535607 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:39.535607 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:39.535607 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:39.535607 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:39.535607 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:39.535607 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:39.535607 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:39.535607 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:39.535607 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:39.535607 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:39.535607 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:39.535607 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:39.536440 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:39.536440 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:39.536440 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:39.536440 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:39.536440 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:39.536440 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:39.536440 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:39.536440 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:39.536440 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:39.536440 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:39.536440 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 23:30:39.536440 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:30:39.537039 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 173/356] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:30:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:39.537339 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59461e8b-ddc9-4eb3-9ab4-18feba145474 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:39.541820 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:39.586318 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a47a6-7033-4844-a3be-1cab79136a7c None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:39.639305 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e7da9-4388-4808-a86d-02cd3a31d26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:39.640749 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e7da9-4388-4808-a86d-02cd3a31d26c None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:39.641033 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e7da9-4388-4808-a86d-02cd3a31d26c None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:39.641312 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e7da9-4388-4808-a86d-02cd3a31d26c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:39.680226 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:39.711542 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59461e8b-ddc9-4eb3-9ab4-18feba145474 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:39.755518 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2e7da9-4388-4808-a86d-02cd3a31d26c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:39.757395 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2e7da9-4388-4808-a86d-02cd3a31d26c None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:39.774302 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7dd784-66a3-407e-8ff8-a21276395f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:39.774942 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7dd784-66a3-407e-8ff8-a21276395f0a None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:39.775375 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7dd784-66a3-407e-8ff8-a21276395f0a None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:39.778382 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7dd784-66a3-407e-8ff8-a21276395f0a None admin] PATH_INFO: `/v3/groups/ac7c419a3715488f9ab314b2075879da` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:39.778382 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7dd784-66a3-407e-8ff8-a21276395f0a None admin] RBAC: Authorizing `identity:get_group(group_id=ac7c419a3715488f9ab314b2075879da)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:39.779251 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7dd784-66a3-407e-8ff8-a21276395f0a None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:39.780910 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 174/357] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Aug 17 23:30:39 2022] GET /identity/v3/groups/ac7c419a3715488f9ab314b2075879da => generated 226 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:39.787060 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a992302-7e2f-4764-b29f-fba2e7d8595a None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:39.787345 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a992302-7e2f-4764-b29f-fba2e7d8595a None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:39.787616 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a992302-7e2f-4764-b29f-fba2e7d8595a None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:39.790732 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 175/358] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:39.792653 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a992302-7e2f-4764-b29f-fba2e7d8595a None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:39.898919 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:39.910989 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 184/359] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:30:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:30:39.911528 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e7da9-4388-4808-a86d-02cd3a31d26c None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:39.948281 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a992302-7e2f-4764-b29f-fba2e7d8595a None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:39.990125 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:39.994592 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35315e3b-001f-4059-933b-aea58d36f567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:39.995211 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35315e3b-001f-4059-933b-aea58d36f567 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:39.995478 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35315e3b-001f-4059-933b-aea58d36f567 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:39.995745 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35315e3b-001f-4059-933b-aea58d36f567 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:39.996812 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35315e3b-001f-4059-933b-aea58d36f567 None admin] RBAC: Adding query filter params (group.id=ac7c419a3715488f9ab314b2075879da, role.id=1182317a465349478dde4e28cb7cf96f, scope.project.id=105a2b33e5ee4cf88b11c48aebf329d7) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:39.997300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35315e3b-001f-4059-933b-aea58d36f567 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:39.998973 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35315e3b-001f-4059-933b-aea58d36f567 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:40.009737 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 176/360] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Aug 17 23:30:39 2022] GET /identity/v3/role_assignments?group.id=ac7c419a3715488f9ab314b2075879da&scope.project.id=105a2b33e5ee4cf88b11c48aebf329d7&role.id=1182317a465349478dde4e28cb7cf96f => generated 260 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:40.021076 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35315e3b-001f-4059-933b-aea58d36f567 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:40.061648 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e7da9-4388-4808-a86d-02cd3a31d26c None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:40.122175 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb11010-2568-47cd-ab99-1c5a3e1a686e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:40.123138 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb11010-2568-47cd-ab99-1c5a3e1a686e None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:40.123586 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb11010-2568-47cd-ab99-1c5a3e1a686e None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:40.124036 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb11010-2568-47cd-ab99-1c5a3e1a686e None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:40.124970 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb11010-2568-47cd-ab99-1c5a3e1a686e None admin] RBAC: Adding query filter params (name=reader) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:40.125412 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb11010-2568-47cd-ab99-1c5a3e1a686e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:40.127105 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb11010-2568-47cd-ab99-1c5a3e1a686e None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:40.137890 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 185/361] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 17 23:30:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:40.138972 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb11010-2568-47cd-ab99-1c5a3e1a686e None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:40.196559 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35315e3b-001f-4059-933b-aea58d36f567 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:40.275985 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd9c95-855d-4b02-b58d-151e955d5612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:40.278127 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd9c95-855d-4b02-b58d-151e955d5612 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:40.278127 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd9c95-855d-4b02-b58d-151e955d5612 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:40.278127 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd9c95-855d-4b02-b58d-151e955d5612 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:40.281359 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:40.302033 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd9c95-855d-4b02-b58d-151e955d5612 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:40.307119 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb11010-2568-47cd-ab99-1c5a3e1a686e None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:40.307433 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-65cd9c95-855d-4b02-b58d-151e955d5612 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 23:30:40.307433 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:40.307433 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:40.307433 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:40.307433 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:40.307433 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:40.307433 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:40.307433 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:40.307433 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:40.307433 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:40.307433 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:40.307433 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:40.307433 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:40.307433 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:40.307433 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:40.307433 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:40.308351 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:40.308351 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:40.308351 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:40.308351 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:40.308351 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:40.308351 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:40.308351 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:40.308351 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:40.308351 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:40.308351 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:40.308351 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:40.308351 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:40.308351 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:40.308351 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:40.308351 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:40.308351 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:40.309191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:40.309191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:40.309191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:40.309191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:40.309191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:40.309191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:40.309191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:40.309191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:40.309191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:40.309191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:40.309191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 23:30:40.309191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:30:40.309191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 177/362] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 17 23:30:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:40.309875 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf4f6c-2f44-4462-9da1-222165b76f79 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:40.310572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf4f6c-2f44-4462-9da1-222165b76f79 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:40.310572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf4f6c-2f44-4462-9da1-222165b76f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:40.333720 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-6fbf4f6c-2f44-4462-9da1-222165b76f79 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:40.333720 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-6fbf4f6c-2f44-4462-9da1-222165b76f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:40.342060 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:40.371692 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1005e-4822-4b84-9984-911d630db96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:40.372814 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1005e-4822-4b84-9984-911d630db96a None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:40.375015 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1005e-4822-4b84-9984-911d630db96a None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:40.375015 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1005e-4822-4b84-9984-911d630db96a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:40.375015 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe1005e-4822-4b84-9984-911d630db96a None admin] RBAC: Adding query filter params (group.id=ac7c419a3715488f9ab314b2075879da, role.id=3428ff8e92274de6817b7e8ee9a56d13, scope.project.id=105a2b33e5ee4cf88b11c48aebf329d7) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:40.375015 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe1005e-4822-4b84-9984-911d630db96a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:40.382079 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe1005e-4822-4b84-9984-911d630db96a None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:40.392129 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 186/363] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Aug 17 23:30:40 2022] GET /identity/v3/role_assignments?group.id=ac7c419a3715488f9ab314b2075879da&scope.project.id=105a2b33e5ee4cf88b11c48aebf329d7&role.id=3428ff8e92274de6817b7e8ee9a56d13 => generated 260 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:40.393575 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1005e-4822-4b84-9984-911d630db96a None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:40.428989 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 178/364] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:40.435161 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf4f6c-2f44-4462-9da1-222165b76f79 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:40.548364 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1005e-4822-4b84-9984-911d630db96a None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:40.597670 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b63eba7-b779-44de-aee5-66500f3d6f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:40.597670 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63eba7-b779-44de-aee5-66500f3d6f98 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:40.597670 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63eba7-b779-44de-aee5-66500f3d6f98 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:40.597670 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63eba7-b779-44de-aee5-66500f3d6f98 None admin] PATH_INFO: `/v3/projects/e959a87f43754d66b9bd795c08d2d827` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:40.598813 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b63eba7-b779-44de-aee5-66500f3d6f98 None admin] RBAC: Authorizing `identity:get_project(project_id=e959a87f43754d66b9bd795c08d2d827)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:40.607003 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b63eba7-b779-44de-aee5-66500f3d6f98 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:40.607003 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 187/365] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:40 2022] GET /identity/v3/projects/e959a87f43754d66b9bd795c08d2d827 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:40.607003 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b63eba7-b779-44de-aee5-66500f3d6f98 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:40.651637 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf4f6c-2f44-4462-9da1-222165b76f79 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:40.678015 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:40.697303 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:40.725899 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24f9633-2261-48b4-a8ea-3295e4e874c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:40.726900 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f9633-2261-48b4-a8ea-3295e4e874c8 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:40.727423 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f9633-2261-48b4-a8ea-3295e4e874c8 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:40.727891 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f9633-2261-48b4-a8ea-3295e4e874c8 None admin] PATH_INFO: `/v3/users/760c062bf340436bb383e31b550c91e5` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:40.729284 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24f9633-2261-48b4-a8ea-3295e4e874c8 None admin] RBAC: Authorizing `identity:get_user(user_id=760c062bf340436bb383e31b550c91e5)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:40.731597 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24f9633-2261-48b4-a8ea-3295e4e874c8 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:40.743988 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 179/366] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:30:40 2022] GET /identity/v3/users/760c062bf340436bb383e31b550c91e5 => generated 293 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:40.745257 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24f9633-2261-48b4-a8ea-3295e4e874c8 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:40.745619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b63eba7-b779-44de-aee5-66500f3d6f98 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:40.797854 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff298f-c60f-4a37-af09-8554ff459976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:40.797854 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff298f-c60f-4a37-af09-8554ff459976 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:40.797854 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff298f-c60f-4a37-af09-8554ff459976 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:40.797854 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff298f-c60f-4a37-af09-8554ff459976 None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:40.803994 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcff298f-c60f-4a37-af09-8554ff459976 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:40.803994 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcff298f-c60f-4a37-af09-8554ff459976 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:40.803994 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcff298f-c60f-4a37-af09-8554ff459976 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:40.829343 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 188/367] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:30:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:40.830302 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff298f-c60f-4a37-af09-8554ff459976 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:40.920412 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24f9633-2261-48b4-a8ea-3295e4e874c8 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:40.967292 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b28a77a-f10e-4214-b041-ad1a42db37ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:40.967616 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28a77a-f10e-4214-b041-ad1a42db37ed None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:40.969752 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28a77a-f10e-4214-b041-ad1a42db37ed None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:40.969752 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28a77a-f10e-4214-b041-ad1a42db37ed None admin] PATH_INFO: `/v3/groups/ac7c419a3715488f9ab314b2075879da` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:40.973425 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b28a77a-f10e-4214-b041-ad1a42db37ed None admin] RBAC: Authorizing `identity:get_group(group_id=ac7c419a3715488f9ab314b2075879da)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:40.974851 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b28a77a-f10e-4214-b041-ad1a42db37ed None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:40.976725 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 180/368] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Aug 17 23:30:40 2022] GET /identity/v3/groups/ac7c419a3715488f9ab314b2075879da => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:40.977393 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b28a77a-f10e-4214-b041-ad1a42db37ed None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:41.009723 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff298f-c60f-4a37-af09-8554ff459976 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:41.027347 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:41.078960 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ac0c7-6d67-45b9-a285-5f48bde064b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:41.079916 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ac0c7-6d67-45b9-a285-5f48bde064b0 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:41.080418 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ac0c7-6d67-45b9-a285-5f48bde064b0 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:41.080915 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ac0c7-6d67-45b9-a285-5f48bde064b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:41.081848 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4ac0c7-6d67-45b9-a285-5f48bde064b0 None admin] RBAC: Adding query filter params (role.id=df3442912b3d43f0a36ebb936b3cb102, user.id=760c062bf340436bb383e31b550c91e5, scope.system=all) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:41.082357 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4ac0c7-6d67-45b9-a285-5f48bde064b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:41.084046 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4ac0c7-6d67-45b9-a285-5f48bde064b0 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:41.103493 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:41.109211 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 189/369] 10.222.0.26 () {56 vars in 1308 bytes} [Wed Aug 17 23:30:40 2022] GET /identity/v3/role_assignments?user.id=760c062bf340436bb383e31b550c91e5&scope.system=all&role.id=df3442912b3d43f0a36ebb936b3cb102 => generated 510 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:41.110225 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ac0c7-6d67-45b9-a285-5f48bde064b0 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:41.161153 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b28a77a-f10e-4214-b041-ad1a42db37ed None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:41.224748 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2323cf3-56e4-4671-ba5d-85c1dd62b740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:41.225376 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2323cf3-56e4-4671-ba5d-85c1dd62b740 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:41.225654 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2323cf3-56e4-4671-ba5d-85c1dd62b740 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:41.225924 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2323cf3-56e4-4671-ba5d-85c1dd62b740 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:41.278624 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ac0c7-6d67-45b9-a285-5f48bde064b0 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:41.329845 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d338a-c6c8-46da-9c62-da5d4df59a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:41.330488 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d338a-c6c8-46da-9c62-da5d4df59a1d None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:41.330769 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d338a-c6c8-46da-9c62-da5d4df59a1d None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:41.331038 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d338a-c6c8-46da-9c62-da5d4df59a1d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:41.344774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d338a-c6c8-46da-9c62-da5d4df59a1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:41.350912 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-5f9d338a-c6c8-46da-9c62-da5d4df59a1d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:30:41.350912 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:41.350912 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:41.350912 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:41.350912 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:41.350912 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:41.350912 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:41.350912 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:41.350912 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:41.350912 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:41.350912 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:41.350912 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:41.350912 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:41.350912 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:41.350912 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:41.350912 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:41.351796 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:41.351796 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:41.351796 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:41.351796 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:41.351796 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:41.351796 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:41.351796 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:41.351796 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:41.351796 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:41.351796 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:41.351796 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:41.351796 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:41.351796 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:41.351796 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:41.351796 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:41.351796 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:41.352625 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:41.352625 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:41.352625 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:41.352625 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:41.352625 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:41.352625 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:41.352625 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:41.352625 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:41.352625 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:41.352625 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:41.352625 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:30:41.352625 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:30:41.352625 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 190/370] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 17 23:30:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:41.353297 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d338a-c6c8-46da-9c62-da5d4df59a1d None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:41.391453 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:41.412303 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2323cf3-56e4-4671-ba5d-85c1dd62b740 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:41.414264 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2323cf3-56e4-4671-ba5d-85c1dd62b740 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:41.476624 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d338a-c6c8-46da-9c62-da5d4df59a1d None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:41.497872 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:41.527286 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a645231c-bdb3-4418-bc5c-d4bd80b395d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:41.530383 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645231c-bdb3-4418-bc5c-d4bd80b395d9 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:41.530951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645231c-bdb3-4418-bc5c-d4bd80b395d9 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:41.531425 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645231c-bdb3-4418-bc5c-d4bd80b395d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:41.532359 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a645231c-bdb3-4418-bc5c-d4bd80b395d9 None admin] RBAC: Adding query filter params (group.id=ac7c419a3715488f9ab314b2075879da, role.id=1182317a465349478dde4e28cb7cf96f, scope.project.id=e959a87f43754d66b9bd795c08d2d827) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:41.532849 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a645231c-bdb3-4418-bc5c-d4bd80b395d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:41.534660 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a645231c-bdb3-4418-bc5c-d4bd80b395d9 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:41.541755 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 191/371] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Aug 17 23:30:41 2022] GET /identity/v3/role_assignments?group.id=ac7c419a3715488f9ab314b2075879da&scope.project.id=e959a87f43754d66b9bd795c08d2d827&role.id=1182317a465349478dde4e28cb7cf96f => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:41.542840 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a645231c-bdb3-4418-bc5c-d4bd80b395d9 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:41.572324 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 181/372] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:30:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:30:41.573755 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2323cf3-56e4-4671-ba5d-85c1dd62b740 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:41.673820 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:41.685347 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a645231c-bdb3-4418-bc5c-d4bd80b395d9 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:41.727927 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2323cf3-56e4-4671-ba5d-85c1dd62b740 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:41.750237 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c8f9d-c825-4dfe-a035-a39dcc773595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:41.751235 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c8f9d-c825-4dfe-a035-a39dcc773595 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:41.751712 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c8f9d-c825-4dfe-a035-a39dcc773595 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:41.752337 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c8f9d-c825-4dfe-a035-a39dcc773595 None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:41.753333 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3c8f9d-c825-4dfe-a035-a39dcc773595 None admin] RBAC: Adding query filter params (name=admin) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:41.753831 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3c8f9d-c825-4dfe-a035-a39dcc773595 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:41.760818 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3c8f9d-c825-4dfe-a035-a39dcc773595 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:41.766975 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f1c76b-65ad-4643-973a-67136e341f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:41.767504 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1c76b-65ad-4643-973a-67136e341f3a None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:41.767788 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1c76b-65ad-4643-973a-67136e341f3a None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:41.770081 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1c76b-65ad-4643-973a-67136e341f3a None admin] PATH_INFO: `/v3/projects/e959a87f43754d66b9bd795c08d2d827` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:41.770081 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f1c76b-65ad-4643-973a-67136e341f3a None admin] RBAC: Authorizing `identity:get_project(project_id=e959a87f43754d66b9bd795c08d2d827)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:41.771996 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f1c76b-65ad-4643-973a-67136e341f3a None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:41.773323 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 182/373] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:41 2022] GET /identity/v3/projects/e959a87f43754d66b9bd795c08d2d827 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:41.781357 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f1c76b-65ad-4643-973a-67136e341f3a None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:41.782664 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 192/374] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:30:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:41.789655 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c8f9d-c825-4dfe-a035-a39dcc773595 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:41.820924 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:41.905577 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f1c76b-65ad-4643-973a-67136e341f3a None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:41.982397 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7686ab-481f-404f-bdc6-03d9ece93d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:41.983040 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7686ab-481f-404f-bdc6-03d9ece93d5a None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:41.983305 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7686ab-481f-404f-bdc6-03d9ece93d5a None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:41.983576 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7686ab-481f-404f-bdc6-03d9ece93d5a None admin] PATH_INFO: `/v3/groups/ac7c419a3715488f9ab314b2075879da` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:41.989295 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7686ab-481f-404f-bdc6-03d9ece93d5a None admin] RBAC: Authorizing `identity:get_group(group_id=ac7c419a3715488f9ab314b2075879da)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:41.990773 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7686ab-481f-404f-bdc6-03d9ece93d5a None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:41.992703 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 183/375] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Aug 17 23:30:41 2022] GET /identity/v3/groups/ac7c419a3715488f9ab314b2075879da => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:42.002545 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7686ab-481f-404f-bdc6-03d9ece93d5a None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:42.053572 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c8f9d-c825-4dfe-a035-a39dcc773595 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:42.123737 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdfd046-b7ee-4c23-b253-d67d261a1c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:42.126149 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdfd046-b7ee-4c23-b253-d67d261a1c0d None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:42.126433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdfd046-b7ee-4c23-b253-d67d261a1c0d None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:42.126705 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdfd046-b7ee-4c23-b253-d67d261a1c0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:42.178107 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:42.196640 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:42.208911 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7686ab-481f-404f-bdc6-03d9ece93d5a None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:42.268944 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdfd046-b7ee-4c23-b253-d67d261a1c0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:42.271637 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdfd046-b7ee-4c23-b253-d67d261a1c0d None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:42.288680 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5526ae0c-7ddd-4409-8db3-9b809f0b998b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:42.289640 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526ae0c-7ddd-4409-8db3-9b809f0b998b None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:42.290220 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526ae0c-7ddd-4409-8db3-9b809f0b998b None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:42.291524 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526ae0c-7ddd-4409-8db3-9b809f0b998b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:42.293749 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5526ae0c-7ddd-4409-8db3-9b809f0b998b None admin] RBAC: Adding query filter params (group.id=ac7c419a3715488f9ab314b2075879da, role.id=3428ff8e92274de6817b7e8ee9a56d13, scope.project.id=e959a87f43754d66b9bd795c08d2d827) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:42.294272 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5526ae0c-7ddd-4409-8db3-9b809f0b998b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:42.295929 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5526ae0c-7ddd-4409-8db3-9b809f0b998b None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:42.311392 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 184/376] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Aug 17 23:30:41 2022] GET /identity/v3/role_assignments?group.id=ac7c419a3715488f9ab314b2075879da&scope.project.id=e959a87f43754d66b9bd795c08d2d827&role.id=3428ff8e92274de6817b7e8ee9a56d13 => generated 260 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:42.386740 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e78ba74-8baa-445b-9e7b-4f9be8722ef6 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:42.387362 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e78ba74-8baa-445b-9e7b-4f9be8722ef6 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:42.387839 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e78ba74-8baa-445b-9e7b-4f9be8722ef6 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:42.391205 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 185/377] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 23:30:42.399699 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a55425-e3d6-4573-8f58-f9a575e500d2 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:42.400368 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a55425-e3d6-4573-8f58-f9a575e500d2 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:42.400843 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a55425-e3d6-4573-8f58-f9a575e500d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:42.451586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-d3a55425-e3d6-4573-8f58-f9a575e500d2 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:42.451586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-d3a55425-e3d6-4573-8f58-f9a575e500d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:42.476355 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 193/378] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:30:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 688 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:30:42.490644 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdfd046-b7ee-4c23-b253-d67d261a1c0d None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:42.566674 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 186/379] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:42.588320 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:42.655018 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdfd046-b7ee-4c23-b253-d67d261a1c0d None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:42.725162 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489025b4-64cf-49f6-b352-9c1825d5ccf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:42.726652 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489025b4-64cf-49f6-b352-9c1825d5ccf4 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:42.727266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489025b4-64cf-49f6-b352-9c1825d5ccf4 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:42.727735 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489025b4-64cf-49f6-b352-9c1825d5ccf4 None admin] PATH_INFO: `/v3/projects/fd475051e9cd456abbb6cff649b374cc` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:42.732677 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489025b4-64cf-49f6-b352-9c1825d5ccf4 None admin] RBAC: Authorizing `identity:get_project(project_id=fd475051e9cd456abbb6cff649b374cc)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:42.734360 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489025b4-64cf-49f6-b352-9c1825d5ccf4 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:42.736579 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 194/380] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:42 2022] GET /identity/v3/projects/fd475051e9cd456abbb6cff649b374cc => generated 344 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:42.742338 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a55425-e3d6-4573-8f58-f9a575e500d2 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:42.866203 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:42.877264 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a55425-e3d6-4573-8f58-f9a575e500d2 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:42.913469 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a6734-60f5-4c56-838d-e84bb2874c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:42.914471 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a6734-60f5-4c56-838d-e84bb2874c9c None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:42.914950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a6734-60f5-4c56-838d-e84bb2874c9c None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:42.915434 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a6734-60f5-4c56-838d-e84bb2874c9c None admin] PATH_INFO: `/v3/groups/3ca1ae290f5c4c6c9fdf541e01401cf0` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:42.923563 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130a6734-60f5-4c56-838d-e84bb2874c9c None admin] RBAC: Authorizing `identity:get_group(group_id=3ca1ae290f5c4c6c9fdf541e01401cf0)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:42.925478 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130a6734-60f5-4c56-838d-e84bb2874c9c None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:42.928025 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 187/381] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Aug 17 23:30:42 2022] GET /identity/v3/groups/3ca1ae290f5c4c6c9fdf541e01401cf0 => generated 229 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:42.934077 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489025b4-64cf-49f6-b352-9c1825d5ccf4 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:42.998962 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a6734-60f5-4c56-838d-e84bb2874c9c None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:43.070856 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489025b4-64cf-49f6-b352-9c1825d5ccf4 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:43.126981 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:43.147248 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674ff165-24cf-4136-b89d-4b1b83d4634b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:43.149275 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ff165-24cf-4136-b89d-4b1b83d4634b None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:43.149763 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ff165-24cf-4136-b89d-4b1b83d4634b None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:43.150405 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ff165-24cf-4136-b89d-4b1b83d4634b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:43.153116 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674ff165-24cf-4136-b89d-4b1b83d4634b None admin] RBAC: Adding query filter params (group.id=3ca1ae290f5c4c6c9fdf541e01401cf0, role.id=33bd60d3b0e94dcbbfe97e87f91a8d78, scope.project.id=fd475051e9cd456abbb6cff649b374cc) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:43.153836 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674ff165-24cf-4136-b89d-4b1b83d4634b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:43.156226 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674ff165-24cf-4136-b89d-4b1b83d4634b None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:43.168620 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 195/382] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Aug 17 23:30:42 2022] GET /identity/v3/role_assignments?group.id=3ca1ae290f5c4c6c9fdf541e01401cf0&scope.project.id=fd475051e9cd456abbb6cff649b374cc&role.id=33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 260 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:43.178209 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd28d4-d127-4f90-a901-943dae22862f None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:43.178787 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd28d4-d127-4f90-a901-943dae22862f None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:43.179406 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd28d4-d127-4f90-a901-943dae22862f None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:43.186396 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 196/383] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:43 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:43.193782 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9584e6-2120-4994-884a-bcc3ce164d1a None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:43.194371 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9584e6-2120-4994-884a-bcc3ce164d1a None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:43.194847 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9584e6-2120-4994-884a-bcc3ce164d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:43.208010 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a6734-60f5-4c56-838d-e84bb2874c9c None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:43.245948 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-ab9584e6-2120-4994-884a-bcc3ce164d1a None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:43.247598 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-ab9584e6-2120-4994-884a-bcc3ce164d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:43.257283 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18e797-b597-4124-a8f2-fbc4f5a87688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:43.257932 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18e797-b597-4124-a8f2-fbc4f5a87688 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:43.258211 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18e797-b597-4124-a8f2-fbc4f5a87688 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:43.258485 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18e797-b597-4124-a8f2-fbc4f5a87688 None admin] PATH_INFO: `/v3/roles/member` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:43.272749 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa18e797-b597-4124-a8f2-fbc4f5a87688 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:43.279726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-fa18e797-b597-4124-a8f2-fbc4f5a87688 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 23:30:43.279726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:43.279726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:43.279726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:43.279726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:43.279726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:43.279726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:43.279726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:43.279726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:43.279726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:43.279726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:43.279726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:43.279726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:43.279726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:43.279726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:43.279726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:43.280686 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:43.280686 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:43.280686 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:43.280686 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:43.280686 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:43.280686 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:43.280686 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:43.280686 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:43.280686 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:43.280686 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:43.280686 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:43.280686 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:43.280686 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:43.280686 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:43.280686 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:43.280686 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:43.281489 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:43.281489 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:43.281489 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:43.281489 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:43.281489 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:43.281489 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:43.281489 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:43.281489 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:43.281489 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:43.281489 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:43.281489 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 23:30:43.281489 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:30:43.287455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 188/384] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:30:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:43.287455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18e797-b597-4124-a8f2-fbc4f5a87688 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:43.315139 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:43.410491 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 197/385] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:43.528601 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18e797-b597-4124-a8f2-fbc4f5a87688 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:43.568535 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd21addb-0616-433f-86f6-523460334a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:43.569155 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21addb-0616-433f-86f6-523460334a1a None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:43.569436 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21addb-0616-433f-86f6-523460334a1a None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:43.569715 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21addb-0616-433f-86f6-523460334a1a None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:43.570392 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd21addb-0616-433f-86f6-523460334a1a None admin] RBAC: Adding query filter params (name=member) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:43.570699 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd21addb-0616-433f-86f6-523460334a1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:43.572104 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd21addb-0616-433f-86f6-523460334a1a None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:43.581497 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 189/386] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 17 23:30:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:43.594487 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9584e6-2120-4994-884a-bcc3ce164d1a None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:43.788087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd21addb-0616-433f-86f6-523460334a1a None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:43.806856 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9584e6-2120-4994-884a-bcc3ce164d1a None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:43.835775 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:43.867221 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb75b00-c576-4811-80c3-72ccd5d9aa51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:43.871097 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb75b00-c576-4811-80c3-72ccd5d9aa51 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:43.871304 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb75b00-c576-4811-80c3-72ccd5d9aa51 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:43.871548 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb75b00-c576-4811-80c3-72ccd5d9aa51 None admin] PATH_INFO: `/v3/groups/ac7c419a3715488f9ab314b2075879da` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:43.872747 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb75b00-c576-4811-80c3-72ccd5d9aa51 None admin] RBAC: Authorizing `identity:get_group(group_id=ac7c419a3715488f9ab314b2075879da)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:43.880014 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb75b00-c576-4811-80c3-72ccd5d9aa51 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:43.880014 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 198/387] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Aug 17 23:30:43 2022] GET /identity/v3/groups/ac7c419a3715488f9ab314b2075879da => generated 226 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:43.891994 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb75b00-c576-4811-80c3-72ccd5d9aa51 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:43.954016 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:44.009640 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb75b00-c576-4811-80c3-72ccd5d9aa51 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:44.019857 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd21addb-0616-433f-86f6-523460334a1a None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:44.054713 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f09126-c82b-49f6-85bf-0c0f2e5dd2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:44.057749 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f09126-c82b-49f6-85bf-0c0f2e5dd2a6 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:44.057749 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f09126-c82b-49f6-85bf-0c0f2e5dd2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:44.057749 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f09126-c82b-49f6-85bf-0c0f2e5dd2a6 None admin] PATH_INFO: `/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:44.058796 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f09126-c82b-49f6-85bf-0c0f2e5dd2a6 None admin] RBAC: Authorizing `identity:get_project(project_id=105a2b33e5ee4cf88b11c48aebf329d7)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:44.060610 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f09126-c82b-49f6-85bf-0c0f2e5dd2a6 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:44.070782 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 199/388] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:43 2022] GET /identity/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:44.072039 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f09126-c82b-49f6-85bf-0c0f2e5dd2a6 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:44.113381 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741449bf-8901-4021-812f-7bcf4d99b34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:44.114393 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741449bf-8901-4021-812f-7bcf4d99b34b None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:44.114882 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741449bf-8901-4021-812f-7bcf4d99b34b None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:44.115412 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741449bf-8901-4021-812f-7bcf4d99b34b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:44.133892 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741449bf-8901-4021-812f-7bcf4d99b34b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:44.140336 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:44.141679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-741449bf-8901-4021-812f-7bcf4d99b34b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 23:30:44.141679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:44.141679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:44.141679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:44.141679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:44.141679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:44.141679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:44.141679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:44.141679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:44.141679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:44.141679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:44.141679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:44.141679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:44.141679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:44.141679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:44.141679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:44.142613 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:44.142613 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:44.142613 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:44.142613 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:44.142613 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:44.142613 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:44.142613 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:44.142613 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:44.142613 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:44.142613 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:44.142613 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:44.142613 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:44.142613 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:44.142613 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:44.142613 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:44.142613 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:44.143449 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:44.143449 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:44.143449 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:44.143449 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:44.143449 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:44.143449 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:44.143449 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:44.143449 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:44.143449 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:44.143449 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:44.143449 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 23:30:44.143449 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:30:44.145576 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 190/389] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 17 23:30:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:44.146809 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5eda97-fd2f-4d7d-ad5c-2d2726a5d498 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:44.147097 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5eda97-fd2f-4d7d-ad5c-2d2726a5d498 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:44.147462 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5eda97-fd2f-4d7d-ad5c-2d2726a5d498 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:44.156029 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 191/390] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:44.156896 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5eda97-fd2f-4d7d-ad5c-2d2726a5d498 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:44.218892 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f09126-c82b-49f6-85bf-0c0f2e5dd2a6 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:44.272166 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c3784-5e8d-4c1e-913b-d71244ba10d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:44.272836 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c3784-5e8d-4c1e-913b-d71244ba10d6 None admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:44.273203 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c3784-5e8d-4c1e-913b-d71244ba10d6 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:44.273484 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c3784-5e8d-4c1e-913b-d71244ba10d6 None admin] PATH_INFO: `/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7/groups/ac7c419a3715488f9ab314b2075879da/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:44.276135 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4c3784-5e8d-4c1e-913b-d71244ba10d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=105a2b33e5ee4cf88b11c48aebf329d7, group_id=ac7c419a3715488f9ab314b2075879da, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:44.277946 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4c3784-5e8d-4c1e-913b-d71244ba10d6 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:44.309958 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 200/391] 10.222.0.26 () {58 vars in 1442 bytes} [Wed Aug 17 23:30:44 2022] PUT /identity/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7/groups/ac7c419a3715488f9ab314b2075879da/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:30:44.316783 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d731e-dafb-4ab9-923d-930ca438c74b None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:44.316783 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d731e-dafb-4ab9-923d-930ca438c74b None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:44.316783 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d731e-dafb-4ab9-923d-930ca438c74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:44.355993 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5eda97-fd2f-4d7d-ad5c-2d2726a5d498 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:44.360298 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-770d731e-dafb-4ab9-923d-930ca438c74b None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:44.364344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-770d731e-dafb-4ab9-923d-930ca438c74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:44.444559 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464925f-79f8-4395-8d12-9d27c2618a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:44.452591 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464925f-79f8-4395-8d12-9d27c2618a29 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:44.452883 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464925f-79f8-4395-8d12-9d27c2618a29 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:44.453158 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464925f-79f8-4395-8d12-9d27c2618a29 None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:44.455004 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c464925f-79f8-4395-8d12-9d27c2618a29 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:44.455004 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c464925f-79f8-4395-8d12-9d27c2618a29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:44.457098 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c464925f-79f8-4395-8d12-9d27c2618a29 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:44.460209 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:44.471018 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 192/392] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:30:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:44.485359 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464925f-79f8-4395-8d12-9d27c2618a29 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:44.550456 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 201/393] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:44.636798 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464925f-79f8-4395-8d12-9d27c2618a29 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:44.680680 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b23d09-e14a-40c6-92a5-da01c85bf5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:44.681322 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b23d09-e14a-40c6-92a5-da01c85bf5d2 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:44.681601 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b23d09-e14a-40c6-92a5-da01c85bf5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:44.681875 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b23d09-e14a-40c6-92a5-da01c85bf5d2 None admin] PATH_INFO: `/v3/groups/ac7c419a3715488f9ab314b2075879da` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:44.683265 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b23d09-e14a-40c6-92a5-da01c85bf5d2 None admin] RBAC: Authorizing `identity:get_group(group_id=ac7c419a3715488f9ab314b2075879da)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:44.685322 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b23d09-e14a-40c6-92a5-da01c85bf5d2 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:44.686894 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 193/394] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Aug 17 23:30:44 2022] GET /identity/v3/groups/ac7c419a3715488f9ab314b2075879da => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:44.693397 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d731e-dafb-4ab9-923d-930ca438c74b None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:44.763944 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b8717-359d-4143-92f4-637167eb6c12 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:44.764294 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b8717-359d-4143-92f4-637167eb6c12 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:44.764518 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b8717-359d-4143-92f4-637167eb6c12 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:44.766095 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 194/395] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:44.773083 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:44.778375 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec788fc-c934-45fa-92c4-5961d8b9cc9d None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:44.778655 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec788fc-c934-45fa-92c4-5961d8b9cc9d None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:44.778931 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec788fc-c934-45fa-92c4-5961d8b9cc9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:44.826882 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-dec788fc-c934-45fa-92c4-5961d8b9cc9d None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:44.830320 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-dec788fc-c934-45fa-92c4-5961d8b9cc9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:44.872392 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d731e-dafb-4ab9-923d-930ca438c74b None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:44.963942 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:44.964129 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dad65a-ec75-458c-af00-3811af32eb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:44.965582 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dad65a-ec75-458c-af00-3811af32eb0c None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:44.966084 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dad65a-ec75-458c-af00-3811af32eb0c None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:44.966562 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dad65a-ec75-458c-af00-3811af32eb0c None admin] PATH_INFO: `/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:44.972036 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dad65a-ec75-458c-af00-3811af32eb0c None admin] RBAC: Authorizing `identity:get_project(project_id=105a2b33e5ee4cf88b11c48aebf329d7)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:44.972167 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 195/396] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:44.981657 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dad65a-ec75-458c-af00-3811af32eb0c None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:44.984121 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 202/397] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:44 2022] GET /identity/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7 => generated 298 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:44.986771 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec788fc-c934-45fa-92c4-5961d8b9cc9d None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:44.996973 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dad65a-ec75-458c-af00-3811af32eb0c None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:45.187780 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec788fc-c934-45fa-92c4-5961d8b9cc9d None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:45.216063 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dad65a-ec75-458c-af00-3811af32eb0c None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:45.229255 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:45.266278 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d463f-8452-4c33-b637-51dc0667f076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:45.266971 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d463f-8452-4c33-b637-51dc0667f076 None admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:45.267252 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d463f-8452-4c33-b637-51dc0667f076 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:45.267525 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d463f-8452-4c33-b637-51dc0667f076 None admin] PATH_INFO: `/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7/groups/ac7c419a3715488f9ab314b2075879da/roles/3428ff8e92274de6817b7e8ee9a56d13` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:45.273379 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714d463f-8452-4c33-b637-51dc0667f076 None admin] RBAC: Authorizing `identity:create_grant(project_id=105a2b33e5ee4cf88b11c48aebf329d7, group_id=ac7c419a3715488f9ab314b2075879da, role_id=3428ff8e92274de6817b7e8ee9a56d13)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:45.274975 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caec0bb6-96ee-4eff-ba45-e06ceca06826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:45.276922 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714d463f-8452-4c33-b637-51dc0667f076 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:45.278985 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec0bb6-96ee-4eff-ba45-e06ceca06826 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:45.279270 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec0bb6-96ee-4eff-ba45-e06ceca06826 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:45.279543 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec0bb6-96ee-4eff-ba45-e06ceca06826 None admin] PATH_INFO: `/v3/roles/member` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:45.295680 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caec0bb6-96ee-4eff-ba45-e06ceca06826 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:45.301337 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-caec0bb6-96ee-4eff-ba45-e06ceca06826 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 23:30:45.301337 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:45.301337 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:45.301337 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:45.301337 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:45.301337 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:45.301337 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:45.301337 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:45.301337 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:45.301337 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:45.301337 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:45.301337 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:45.301337 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:45.301337 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:45.301337 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:45.301337 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:45.302317 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:45.302317 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:45.302317 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:45.302317 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:45.302317 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:45.302317 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:45.302317 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:45.302317 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:45.302317 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:45.302317 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:45.302317 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:45.302317 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:45.302317 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:45.302317 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:45.302317 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:45.302317 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:45.303719 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:45.303719 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:45.303719 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:45.303719 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:45.303719 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:45.303719 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:45.303719 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:45.303719 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:45.303719 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:45.303719 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:45.303719 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 23:30:45.303719 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:30:45.304358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 203/398] 10.222.0.26 () {58 vars in 1442 bytes} [Wed Aug 17 23:30:44 2022] PUT /identity/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7/groups/ac7c419a3715488f9ab314b2075879da/roles/3428ff8e92274de6817b7e8ee9a56d13 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:30:45.304358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d463f-8452-4c33-b637-51dc0667f076 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:45.306797 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 196/399] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:30:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:45.312778 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caec0bb6-96ee-4eff-ba45-e06ceca06826 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:45.440595 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:45.505237 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caec0bb6-96ee-4eff-ba45-e06ceca06826 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:45.548771 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0917bd57-1637-4d26-9359-1b1f11d46e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:45.549786 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917bd57-1637-4d26-9359-1b1f11d46e53 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:45.550277 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917bd57-1637-4d26-9359-1b1f11d46e53 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:45.550750 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917bd57-1637-4d26-9359-1b1f11d46e53 None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:45.551628 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0917bd57-1637-4d26-9359-1b1f11d46e53 None admin] RBAC: Adding query filter params (name=member) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:45.552189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0917bd57-1637-4d26-9359-1b1f11d46e53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:45.553783 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0917bd57-1637-4d26-9359-1b1f11d46e53 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:45.570382 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 197/400] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 17 23:30:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:45.571651 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ed6c3-18d7-4a58-b41c-de9555f11625 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:45.575705 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ed6c3-18d7-4a58-b41c-de9555f11625 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:45.575705 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ed6c3-18d7-4a58-b41c-de9555f11625 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:45.575705 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 198/401] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:45.579067 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee24faf0-a4d6-4a54-9f60-3751bc6edf0a None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:45.579348 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee24faf0-a4d6-4a54-9f60-3751bc6edf0a None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:45.579623 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee24faf0-a4d6-4a54-9f60-3751bc6edf0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:45.582993 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d463f-8452-4c33-b637-51dc0667f076 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:45.599276 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-ee24faf0-a4d6-4a54-9f60-3751bc6edf0a None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:45.600561 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-ee24faf0-a4d6-4a54-9f60-3751bc6edf0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:45.636395 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8036d7-6ab7-455d-a3a7-5b68dc19525d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:45.637883 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8036d7-6ab7-455d-a3a7-5b68dc19525d None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:45.638755 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8036d7-6ab7-455d-a3a7-5b68dc19525d None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:45.639474 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8036d7-6ab7-455d-a3a7-5b68dc19525d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:45.662390 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8036d7-6ab7-455d-a3a7-5b68dc19525d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:45.673587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-ab8036d7-6ab7-455d-a3a7-5b68dc19525d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 23:30:45.673587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:45.673587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:45.673587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:45.673587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:45.673587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:45.673587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:45.673587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:45.673587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:45.673587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:45.673587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:45.673587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:45.673587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:45.673587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:45.673587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:45.673587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:45.674511 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:45.674511 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:45.674511 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:45.674511 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:45.674511 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:45.674511 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:45.674511 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:45.674511 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:45.674511 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:45.674511 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:45.674511 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:45.674511 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:45.674511 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:45.674511 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:45.674511 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:45.674511 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:45.675287 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:45.675287 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:45.675287 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:45.675287 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:45.675287 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:45.675287 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:45.675287 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:45.675287 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:45.675287 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:45.675287 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:45.675287 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 23:30:45.675287 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:30:45.682968 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 204/402] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 17 23:30:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:45.685649 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8036d7-6ab7-455d-a3a7-5b68dc19525d None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:45.709217 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 199/403] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 23:30:45.711022 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee24faf0-a4d6-4a54-9f60-3751bc6edf0a None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:45.739497 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:45.787021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:45.851666 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee24faf0-a4d6-4a54-9f60-3751bc6edf0a None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:45.890597 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c519d7d-9149-4a31-962b-3f53e63339cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:45.891560 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c519d7d-9149-4a31-962b-3f53e63339cd None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:45.892097 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c519d7d-9149-4a31-962b-3f53e63339cd None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:45.892688 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c519d7d-9149-4a31-962b-3f53e63339cd None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:45.893590 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c519d7d-9149-4a31-962b-3f53e63339cd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:45.894155 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c519d7d-9149-4a31-962b-3f53e63339cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:45.896578 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c519d7d-9149-4a31-962b-3f53e63339cd None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:45.901292 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8036d7-6ab7-455d-a3a7-5b68dc19525d None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:45.906995 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 200/404] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:30:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:45.914739 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c519d7d-9149-4a31-962b-3f53e63339cd None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:45.983501 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5836e54-b0eb-4a5b-bf48-e3a151a64288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:45.984188 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5836e54-b0eb-4a5b-bf48-e3a151a64288 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:45.984471 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5836e54-b0eb-4a5b-bf48-e3a151a64288 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:45.984743 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5836e54-b0eb-4a5b-bf48-e3a151a64288 None admin] PATH_INFO: `/v3/groups/ac7c419a3715488f9ab314b2075879da` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:45.985967 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5836e54-b0eb-4a5b-bf48-e3a151a64288 None admin] RBAC: Authorizing `identity:get_group(group_id=ac7c419a3715488f9ab314b2075879da)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:45.987525 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5836e54-b0eb-4a5b-bf48-e3a151a64288 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:45.997895 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 205/405] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Aug 17 23:30:45 2022] GET /identity/v3/groups/ac7c419a3715488f9ab314b2075879da => generated 226 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:46.006558 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5836e54-b0eb-4a5b-bf48-e3a151a64288 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:46.034763 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c519d7d-9149-4a31-962b-3f53e63339cd None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:46.046923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:46.075868 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e31b36-613c-4a76-af57-b12cce192b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:46.076957 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e31b36-613c-4a76-af57-b12cce192b7c None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:46.077432 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e31b36-613c-4a76-af57-b12cce192b7c None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:46.077956 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e31b36-613c-4a76-af57-b12cce192b7c None admin] PATH_INFO: `/v3/groups/ac7c419a3715488f9ab314b2075879da` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:46.079409 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e31b36-613c-4a76-af57-b12cce192b7c None admin] RBAC: Authorizing `identity:get_group(group_id=ac7c419a3715488f9ab314b2075879da)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:46.081083 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e31b36-613c-4a76-af57-b12cce192b7c None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:46.083893 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 201/406] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Aug 17 23:30:45 2022] GET /identity/v3/groups/ac7c419a3715488f9ab314b2075879da => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:46.090671 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e31b36-613c-4a76-af57-b12cce192b7c None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:46.185662 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:46.190402 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5836e54-b0eb-4a5b-bf48-e3a151a64288 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:46.209784 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e31b36-613c-4a76-af57-b12cce192b7c None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:46.232789 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0af6e2c-7ba8-420b-bb1e-552c4db7a337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:46.232789 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af6e2c-7ba8-420b-bb1e-552c4db7a337 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:46.233485 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af6e2c-7ba8-420b-bb1e-552c4db7a337 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:46.233774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af6e2c-7ba8-420b-bb1e-552c4db7a337 None admin] PATH_INFO: `/v3/projects/e959a87f43754d66b9bd795c08d2d827` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:46.238765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0af6e2c-7ba8-420b-bb1e-552c4db7a337 None admin] RBAC: Authorizing `identity:get_project(project_id=e959a87f43754d66b9bd795c08d2d827)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:46.240247 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0af6e2c-7ba8-420b-bb1e-552c4db7a337 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:46.243686 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 206/407] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:46 2022] GET /identity/v3/projects/e959a87f43754d66b9bd795c08d2d827 => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:46.243686 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0af6e2c-7ba8-420b-bb1e-552c4db7a337 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:46.278060 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350cfa7f-d83c-455d-8fb9-c8ced055236a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:46.278708 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350cfa7f-d83c-455d-8fb9-c8ced055236a None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:46.278985 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350cfa7f-d83c-455d-8fb9-c8ced055236a None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:46.279255 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350cfa7f-d83c-455d-8fb9-c8ced055236a None admin] PATH_INFO: `/v3/projects/e959a87f43754d66b9bd795c08d2d827` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:46.280441 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350cfa7f-d83c-455d-8fb9-c8ced055236a None admin] RBAC: Authorizing `identity:get_project(project_id=e959a87f43754d66b9bd795c08d2d827)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:46.281951 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350cfa7f-d83c-455d-8fb9-c8ced055236a None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:46.292411 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 202/408] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:46 2022] GET /identity/v3/projects/e959a87f43754d66b9bd795c08d2d827 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:46.294025 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350cfa7f-d83c-455d-8fb9-c8ced055236a None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:46.362647 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:46.467696 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0af6e2c-7ba8-420b-bb1e-552c4db7a337 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:46.483931 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350cfa7f-d83c-455d-8fb9-c8ced055236a None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:46.538160 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aada9a7d-14b5-4d16-9e8d-1030346322a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:46.538796 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada9a7d-14b5-4d16-9e8d-1030346322a9 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:46.539072 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada9a7d-14b5-4d16-9e8d-1030346322a9 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:46.539339 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada9a7d-14b5-4d16-9e8d-1030346322a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:46.550369 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4402a748-05f5-4f67-a218-a1682edab62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:46.551039 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402a748-05f5-4f67-a218-a1682edab62a None admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:46.551326 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402a748-05f5-4f67-a218-a1682edab62a None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:46.551596 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402a748-05f5-4f67-a218-a1682edab62a None admin] PATH_INFO: `/v3/projects/e959a87f43754d66b9bd795c08d2d827/groups/ac7c419a3715488f9ab314b2075879da/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:46.555043 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4402a748-05f5-4f67-a218-a1682edab62a None admin] RBAC: Authorizing `identity:create_grant(project_id=e959a87f43754d66b9bd795c08d2d827, group_id=ac7c419a3715488f9ab314b2075879da, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:46.556544 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4402a748-05f5-4f67-a218-a1682edab62a None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:46.563731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aada9a7d-14b5-4d16-9e8d-1030346322a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:46.571953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-aada9a7d-14b5-4d16-9e8d-1030346322a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:30:46.571953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:46.571953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:46.571953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:46.571953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:46.571953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:46.571953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:46.571953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:46.571953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:46.571953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:46.571953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:46.571953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:46.571953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:46.571953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:46.571953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:46.571953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:46.572906 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:46.572906 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:46.572906 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:46.572906 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:46.572906 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:46.572906 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:46.572906 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:46.572906 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:46.572906 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:46.572906 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:46.572906 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:46.572906 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:46.572906 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:46.572906 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:46.572906 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:46.572906 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:46.573716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:46.573716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:46.573716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:46.573716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:46.573716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:46.573716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:46.573716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:46.573716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:46.573716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:46.573716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:46.573716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:30:46.573716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:30:46.579558 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 207/409] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 17 23:30:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:46.580893 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aada9a7d-14b5-4d16-9e8d-1030346322a9 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:46.594516 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 203/410] 10.222.0.26 () {58 vars in 1442 bytes} [Wed Aug 17 23:30:46 2022] PUT /identity/v3/projects/e959a87f43754d66b9bd795c08d2d827/groups/ac7c419a3715488f9ab314b2075879da/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:30:46.594516 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4402a748-05f5-4f67-a218-a1682edab62a None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:46.645879 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:46.753482 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aada9a7d-14b5-4d16-9e8d-1030346322a9 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:46.763157 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4402a748-05f5-4f67-a218-a1682edab62a None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:46.776061 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:46.813476 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0366925-9ddf-4bf3-8cca-18ab3ca2ce40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:46.814152 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0366925-9ddf-4bf3-8cca-18ab3ca2ce40 None admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:46.814542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0366925-9ddf-4bf3-8cca-18ab3ca2ce40 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:46.814878 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0366925-9ddf-4bf3-8cca-18ab3ca2ce40 None admin] PATH_INFO: `/v3/projects/e959a87f43754d66b9bd795c08d2d827/groups/ac7c419a3715488f9ab314b2075879da/roles/3428ff8e92274de6817b7e8ee9a56d13` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:46.818771 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d95042-6049-4a4f-93e6-43cb898b4fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:46.818898 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0366925-9ddf-4bf3-8cca-18ab3ca2ce40 None admin] RBAC: Authorizing `identity:create_grant(project_id=e959a87f43754d66b9bd795c08d2d827, group_id=ac7c419a3715488f9ab314b2075879da, role_id=3428ff8e92274de6817b7e8ee9a56d13)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:46.819382 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d95042-6049-4a4f-93e6-43cb898b4fd2 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:46.819663 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d95042-6049-4a4f-93e6-43cb898b4fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:46.819936 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d95042-6049-4a4f-93e6-43cb898b4fd2 None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:46.820356 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0366925-9ddf-4bf3-8cca-18ab3ca2ce40 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:46.820638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d95042-6049-4a4f-93e6-43cb898b4fd2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:46.820941 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d95042-6049-4a4f-93e6-43cb898b4fd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:46.822382 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d95042-6049-4a4f-93e6-43cb898b4fd2 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:46.833247 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 204/411] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:30:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:46.835516 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14445823-1234-4141-9129-34d82ea77195 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:46.835516 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14445823-1234-4141-9129-34d82ea77195 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:46.835841 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14445823-1234-4141-9129-34d82ea77195 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:46.837543 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 205/412] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:46.843065 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14445823-1234-4141-9129-34d82ea77195 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:46.853408 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 208/413] 10.222.0.26 () {58 vars in 1442 bytes} [Wed Aug 17 23:30:46 2022] PUT /identity/v3/projects/e959a87f43754d66b9bd795c08d2d827/groups/ac7c419a3715488f9ab314b2075879da/roles/3428ff8e92274de6817b7e8ee9a56d13 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:30:46.855816 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff360a-43c0-4543-970d-5614d49ad929 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:46.855816 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff360a-43c0-4543-970d-5614d49ad929 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:46.863708 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff360a-43c0-4543-970d-5614d49ad929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:46.897557 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-65ff360a-43c0-4543-970d-5614d49ad929 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:46.903986 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-65ff360a-43c0-4543-970d-5614d49ad929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:46.993265 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14445823-1234-4141-9129-34d82ea77195 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:47.067949 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 209/414] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:47.068676 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d9522-8107-4df1-9791-eebbc1084d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:47.069351 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d9522-8107-4df1-9791-eebbc1084d1b None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:47.070332 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d9522-8107-4df1-9791-eebbc1084d1b None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:47.070610 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d9522-8107-4df1-9791-eebbc1084d1b None admin] PATH_INFO: `/v3/groups/3ca1ae290f5c4c6c9fdf541e01401cf0` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:47.071796 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61d9522-8107-4df1-9791-eebbc1084d1b None admin] RBAC: Authorizing `identity:get_group(group_id=3ca1ae290f5c4c6c9fdf541e01401cf0)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:47.073306 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61d9522-8107-4df1-9791-eebbc1084d1b None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:47.075744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 206/415] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Aug 17 23:30:46 2022] GET /identity/v3/groups/3ca1ae290f5c4c6c9fdf541e01401cf0 => generated 229 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:47.082730 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ff360a-43c0-4543-970d-5614d49ad929 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:47.114972 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:47.201959 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ff360a-43c0-4543-970d-5614d49ad929 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:47.247989 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a026df26-b6bd-425d-88d8-54bc6225cd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:47.247989 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026df26-b6bd-425d-88d8-54bc6225cd48 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:47.247989 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026df26-b6bd-425d-88d8-54bc6225cd48 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:47.247989 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026df26-b6bd-425d-88d8-54bc6225cd48 None admin] PATH_INFO: `/v3/projects/fd475051e9cd456abbb6cff649b374cc` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:47.256007 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a026df26-b6bd-425d-88d8-54bc6225cd48 None admin] RBAC: Authorizing `identity:get_project(project_id=fd475051e9cd456abbb6cff649b374cc)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:47.256007 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a026df26-b6bd-425d-88d8-54bc6225cd48 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:47.256007 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 210/416] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:47 2022] GET /identity/v3/projects/fd475051e9cd456abbb6cff649b374cc => generated 344 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:47.267988 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d9522-8107-4df1-9791-eebbc1084d1b None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:47.310294 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:47.380699 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d9522-8107-4df1-9791-eebbc1084d1b None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:47.416371 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ea914-5fc9-4eb9-a603-b2ed523c3def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:47.417046 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ea914-5fc9-4eb9-a603-b2ed523c3def None admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:47.417376 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ea914-5fc9-4eb9-a603-b2ed523c3def None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:47.419994 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ea914-5fc9-4eb9-a603-b2ed523c3def None admin] PATH_INFO: `/v3/projects/fd475051e9cd456abbb6cff649b374cc/groups/3ca1ae290f5c4c6c9fdf541e01401cf0/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:47.419994 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20ea914-5fc9-4eb9-a603-b2ed523c3def None admin] RBAC: Authorizing `identity:create_grant(project_id=fd475051e9cd456abbb6cff649b374cc, group_id=3ca1ae290f5c4c6c9fdf541e01401cf0, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:47.422283 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20ea914-5fc9-4eb9-a603-b2ed523c3def None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:47.439343 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a026df26-b6bd-425d-88d8-54bc6225cd48 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:47.448164 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 207/417] 10.222.0.26 () {58 vars in 1442 bytes} [Wed Aug 17 23:30:47 2022] PUT /identity/v3/projects/fd475051e9cd456abbb6cff649b374cc/groups/3ca1ae290f5c4c6c9fdf541e01401cf0/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:30:47.552006 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:47.570445 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a026df26-b6bd-425d-88d8-54bc6225cd48 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:47.609933 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea186e05-6bf7-435e-8add-8d8a81406d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:47.610572 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea186e05-6bf7-435e-8add-8d8a81406d06 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:47.610848 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea186e05-6bf7-435e-8add-8d8a81406d06 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:47.611118 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea186e05-6bf7-435e-8add-8d8a81406d06 None admin] PATH_INFO: `/v3/roles/member` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:47.625215 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea186e05-6bf7-435e-8add-8d8a81406d06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:47.631452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-ea186e05-6bf7-435e-8add-8d8a81406d06 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 23:30:47.631452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:47.631452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:47.631452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:47.631452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:47.631452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:47.631452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:47.631452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:47.631452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:47.631452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:47.631452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:47.631452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:47.631452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:47.631452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:47.631452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:47.631452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:47.632425 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:47.632425 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:47.632425 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:47.632425 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:47.632425 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:47.632425 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:47.632425 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:47.632425 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:47.632425 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:47.632425 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:47.632425 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:47.632425 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:47.632425 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:47.632425 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:47.632425 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:47.632425 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:47.633207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:47.633207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:47.633207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:47.633207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:47.633207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:47.633207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:47.633207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:47.633207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:47.633207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:47.633207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:47.633207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 23:30:47.633207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:30:47.634517 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 211/418] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:30:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:47.644007 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ea914-5fc9-4eb9-a603-b2ed523c3def None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:47.707397 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319e2ec-dc01-4910-b6bc-bbce4cb80172 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:47.707625 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319e2ec-dc01-4910-b6bc-bbce4cb80172 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:47.707862 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319e2ec-dc01-4910-b6bc-bbce4cb80172 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:47.709480 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 212/419] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 23:30:47.716255 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6332f6-2df0-4b28-b2c1-8e931409aba9 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:47.716255 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6332f6-2df0-4b28-b2c1-8e931409aba9 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:47.716255 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6332f6-2df0-4b28-b2c1-8e931409aba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:47.745502 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-5e6332f6-2df0-4b28-b2c1-8e931409aba9 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:47.749795 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-5e6332f6-2df0-4b28-b2c1-8e931409aba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:47.784896 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:47.791580 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ea914-5fc9-4eb9-a603-b2ed523c3def None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:47.837956 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f62b492-7d7d-4aae-9d17-dc91f01b99a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:47.839133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62b492-7d7d-4aae-9d17-dc91f01b99a7 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:47.840824 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62b492-7d7d-4aae-9d17-dc91f01b99a7 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:47.841562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62b492-7d7d-4aae-9d17-dc91f01b99a7 None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:47.842473 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f62b492-7d7d-4aae-9d17-dc91f01b99a7 None admin] RBAC: Adding query filter params (name=member) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:47.842981 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f62b492-7d7d-4aae-9d17-dc91f01b99a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:47.844731 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f62b492-7d7d-4aae-9d17-dc91f01b99a7 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:47.864291 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 208/420] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 17 23:30:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:47.876849 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f62b492-7d7d-4aae-9d17-dc91f01b99a7 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:47.929956 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 213/421] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:48.015142 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f62b492-7d7d-4aae-9d17-dc91f01b99a7 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:48.072070 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef172b4-6cd2-46b9-af2c-a2070864bac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:48.073075 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef172b4-6cd2-46b9-af2c-a2070864bac1 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:48.073587 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef172b4-6cd2-46b9-af2c-a2070864bac1 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:48.074618 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef172b4-6cd2-46b9-af2c-a2070864bac1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:48.128340 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:48.233089 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef172b4-6cd2-46b9-af2c-a2070864bac1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:48.234562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef172b4-6cd2-46b9-af2c-a2070864bac1 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:48.370276 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6332f6-2df0-4b28-b2c1-8e931409aba9 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:48.462109 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 209/422] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:30:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 17 23:30:48.469140 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef172b4-6cd2-46b9-af2c-a2070864bac1 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:48.516706 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:48.573637 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6332f6-2df0-4b28-b2c1-8e931409aba9 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:48.586775 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:48.678369 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c3d28-4d95-47b1-9f7c-64111f1d1fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:48.678939 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c3d28-4d95-47b1-9f7c-64111f1d1fb3 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:48.679207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c3d28-4d95-47b1-9f7c-64111f1d1fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:48.679469 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c3d28-4d95-47b1-9f7c-64111f1d1fb3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:48.706464 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c3d28-4d95-47b1-9f7c-64111f1d1fb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:48.711419 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-ec2c3d28-4d95-47b1-9f7c-64111f1d1fb3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 23:30:48.711419 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:48.711419 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:48.711419 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:48.711419 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:48.711419 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:48.711419 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:48.711419 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:48.711419 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:48.711419 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:48.711419 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:48.711419 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:48.711419 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:48.711419 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:48.711419 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:48.711419 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:48.712327 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:48.712327 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:48.712327 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:48.712327 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:48.712327 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:48.712327 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:48.712327 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:48.712327 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:48.712327 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:48.712327 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:48.712327 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:48.712327 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:48.712327 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:48.712327 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:48.712327 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:48.712327 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:48.713145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:48.713145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:48.713145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:48.713145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:48.713145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:48.713145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:48.713145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:48.713145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:48.713145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:48.713145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:48.713145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 23:30:48.713145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:30:48.715835 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 214/423] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 17 23:30:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:48.727818 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c3d28-4d95-47b1-9f7c-64111f1d1fb3 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:48.729299 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef172b4-6cd2-46b9-af2c-a2070864bac1 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:48.825841 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6309d7c-1f0b-464a-9eea-b15beaf1579b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:48.827724 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6309d7c-1f0b-464a-9eea-b15beaf1579b None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:48.827724 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6309d7c-1f0b-464a-9eea-b15beaf1579b None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:48.827724 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6309d7c-1f0b-464a-9eea-b15beaf1579b None admin] PATH_INFO: `/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:48.829248 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6309d7c-1f0b-464a-9eea-b15beaf1579b None admin] RBAC: Authorizing `identity:get_project(project_id=105a2b33e5ee4cf88b11c48aebf329d7)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:48.835986 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6309d7c-1f0b-464a-9eea-b15beaf1579b None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:48.835986 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 210/424] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:48 2022] GET /identity/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7 => generated 298 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:48.839121 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6309d7c-1f0b-464a-9eea-b15beaf1579b None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:48.960696 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c3d28-4d95-47b1-9f7c-64111f1d1fb3 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:49.019242 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a58de2-5b4a-4bea-b8cb-fe5b587a1f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:49.020018 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a58de2-5b4a-4bea-b8cb-fe5b587a1f92 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:49.020308 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a58de2-5b4a-4bea-b8cb-fe5b587a1f92 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:49.021495 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a58de2-5b4a-4bea-b8cb-fe5b587a1f92 None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:49.021495 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a58de2-5b4a-4bea-b8cb-fe5b587a1f92 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:49.021802 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a58de2-5b4a-4bea-b8cb-fe5b587a1f92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:49.023084 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a58de2-5b4a-4bea-b8cb-fe5b587a1f92 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:49.025995 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:49.035157 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 215/425] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:30:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:49.041118 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a58de2-5b4a-4bea-b8cb-fe5b587a1f92 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:49.109921 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6309d7c-1f0b-464a-9eea-b15beaf1579b None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:49.166416 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a58de2-5b4a-4bea-b8cb-fe5b587a1f92 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:49.183453 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:49.198436 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c0440-99d8-4a17-8d97-45658729dd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:49.200421 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c0440-99d8-4a17-8d97-45658729dd19 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:49.200713 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c0440-99d8-4a17-8d97-45658729dd19 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:49.200996 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c0440-99d8-4a17-8d97-45658729dd19 None admin] PATH_INFO: `/v3/groups/ac7c419a3715488f9ab314b2075879da` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:49.207982 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02c0440-99d8-4a17-8d97-45658729dd19 None admin] RBAC: Authorizing `identity:get_group(group_id=ac7c419a3715488f9ab314b2075879da)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:49.207982 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02c0440-99d8-4a17-8d97-45658729dd19 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:49.207982 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 211/426] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Aug 17 23:30:48 2022] GET /identity/v3/groups/ac7c419a3715488f9ab314b2075879da => generated 226 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:49.209383 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff3c095-025b-4d4b-8893-143cc6eaa20b None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:49.209664 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff3c095-025b-4d4b-8893-143cc6eaa20b None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:49.209945 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff3c095-025b-4d4b-8893-143cc6eaa20b None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:49.213014 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427790f9-b331-4af4-9061-f1450e7576e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:49.213659 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427790f9-b331-4af4-9061-f1450e7576e0 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:49.213995 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427790f9-b331-4af4-9061-f1450e7576e0 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:49.214266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427790f9-b331-4af4-9061-f1450e7576e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:49.220249 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 212/427] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:49 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:49.221117 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff3c095-025b-4d4b-8893-143cc6eaa20b None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:49.305864 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:49.317757 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427790f9-b331-4af4-9061-f1450e7576e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:49.319263 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427790f9-b331-4af4-9061-f1450e7576e0 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:49.393701 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff3c095-025b-4d4b-8893-143cc6eaa20b None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:49.444113 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c12fda-f813-41ab-b64b-7c7c11b9316b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:49.444969 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c12fda-f813-41ab-b64b-7c7c11b9316b None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:49.445520 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c12fda-f813-41ab-b64b-7c7c11b9316b None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:49.446017 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c12fda-f813-41ab-b64b-7c7c11b9316b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:49.446958 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c12fda-f813-41ab-b64b-7c7c11b9316b None admin] RBAC: Adding query filter params (group.id=ac7c419a3715488f9ab314b2075879da, role.id=1182317a465349478dde4e28cb7cf96f, scope.project.id=105a2b33e5ee4cf88b11c48aebf329d7) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:49.447480 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c12fda-f813-41ab-b64b-7c7c11b9316b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:49.449160 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c12fda-f813-41ab-b64b-7c7c11b9316b None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:49.464061 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 213/428] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Aug 17 23:30:49 2022] GET /identity/v3/role_assignments?group.id=ac7c419a3715488f9ab314b2075879da&scope.project.id=105a2b33e5ee4cf88b11c48aebf329d7&role.id=1182317a465349478dde4e28cb7cf96f => generated 611 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:49.464061 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0493146-326d-461d-9fcc-669d8564edc5 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:49.464061 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0493146-326d-461d-9fcc-669d8564edc5 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:49.464061 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0493146-326d-461d-9fcc-669d8564edc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:49.476799 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 216/429] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:30:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:30:49.480267 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa240a0-6a3b-432c-a07a-ca1f86e8cc43 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:49.480930 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa240a0-6a3b-432c-a07a-ca1f86e8cc43 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:49.483023 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa240a0-6a3b-432c-a07a-ca1f86e8cc43 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:49.486580 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 217/430] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:49 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:49.488793 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa240a0-6a3b-432c-a07a-ca1f86e8cc43 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:49.509228 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-e0493146-326d-461d-9fcc-669d8564edc5 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:49.510521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-e0493146-326d-461d-9fcc-669d8564edc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:49.637011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:49.664281 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 214/431] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:49.666011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb8217-eeb0-4217-a7ea-e2860a74dbd9 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:49.666498 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb8217-eeb0-4217-a7ea-e2860a74dbd9 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:49.666966 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb8217-eeb0-4217-a7ea-e2860a74dbd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:49.679275 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:49.689997 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa240a0-6a3b-432c-a07a-ca1f86e8cc43 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:49.694516 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-6fdb8217-eeb0-4217-a7ea-e2860a74dbd9 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:49.696037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-6fdb8217-eeb0-4217-a7ea-e2860a74dbd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:49.739685 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f78c548-03ed-4a43-8eb7-08b0de4acd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:49.739685 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f78c548-03ed-4a43-8eb7-08b0de4acd1f None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:49.740032 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f78c548-03ed-4a43-8eb7-08b0de4acd1f None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:49.742514 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f78c548-03ed-4a43-8eb7-08b0de4acd1f None admin] PATH_INFO: `/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:49.742514 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f78c548-03ed-4a43-8eb7-08b0de4acd1f None admin] RBAC: Authorizing `identity:get_project(project_id=105a2b33e5ee4cf88b11c48aebf329d7)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:49.744040 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f78c548-03ed-4a43-8eb7-08b0de4acd1f None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:49.747716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 218/432] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:49 2022] GET /identity/v3/projects/105a2b33e5ee4cf88b11c48aebf329d7 => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:49.754538 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f78c548-03ed-4a43-8eb7-08b0de4acd1f None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:49.837832 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 215/433] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:49.912480 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f78c548-03ed-4a43-8eb7-08b0de4acd1f None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:49.922202 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02d5b6-8722-4dc8-8916-425cac64647e None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:49.922492 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02d5b6-8722-4dc8-8916-425cac64647e None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:49.922762 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02d5b6-8722-4dc8-8916-425cac64647e None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:49.924330 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 216/434] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 23:30:49.929336 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3678540-fc7b-4859-9f58-074e67846982 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:49.929872 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3678540-fc7b-4859-9f58-074e67846982 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:49.930361 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3678540-fc7b-4859-9f58-074e67846982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:49.954917 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d90f6df-43a7-45eb-b470-a16173909a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:49.954917 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90f6df-43a7-45eb-b470-a16173909a51 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:49.954917 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90f6df-43a7-45eb-b470-a16173909a51 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:49.955227 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90f6df-43a7-45eb-b470-a16173909a51 None admin] PATH_INFO: `/v3/groups/ac7c419a3715488f9ab314b2075879da` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:49.956373 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d90f6df-43a7-45eb-b470-a16173909a51 None admin] RBAC: Authorizing `identity:get_group(group_id=ac7c419a3715488f9ab314b2075879da)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:49.960186 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d90f6df-43a7-45eb-b470-a16173909a51 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:49.962625 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 219/435] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Aug 17 23:30:49 2022] GET /identity/v3/groups/ac7c419a3715488f9ab314b2075879da => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:49.963879 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-d3678540-fc7b-4859-9f58-074e67846982 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:49.965740 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-d3678540-fc7b-4859-9f58-074e67846982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:49.968362 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d90f6df-43a7-45eb-b470-a16173909a51 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:50.007580 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:50.084942 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 217/436] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:50.086604 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3678540-fc7b-4859-9f58-074e67846982 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:50.134091 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d90f6df-43a7-45eb-b470-a16173909a51 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:50.164699 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:50.181971 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6101b-7368-458a-abe7-3ca3e60211a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:50.182999 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6101b-7368-458a-abe7-3ca3e60211a1 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:50.184022 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6101b-7368-458a-abe7-3ca3e60211a1 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:50.184454 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6101b-7368-458a-abe7-3ca3e60211a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:50.185222 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f6101b-7368-458a-abe7-3ca3e60211a1 None admin] RBAC: Adding query filter params (group.id=ac7c419a3715488f9ab314b2075879da, role.id=3428ff8e92274de6817b7e8ee9a56d13, scope.project.id=105a2b33e5ee4cf88b11c48aebf329d7) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:50.185533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f6101b-7368-458a-abe7-3ca3e60211a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:50.186976 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f6101b-7368-458a-abe7-3ca3e60211a1 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:50.200060 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 220/437] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Aug 17 23:30:49 2022] GET /identity/v3/role_assignments?group.id=ac7c419a3715488f9ab314b2075879da&scope.project.id=105a2b33e5ee4cf88b11c48aebf329d7&role.id=3428ff8e92274de6817b7e8ee9a56d13 => generated 611 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:30:50.200060 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6101b-7368-458a-abe7-3ca3e60211a1 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:50.259517 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3678540-fc7b-4859-9f58-074e67846982 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:50.331886 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5c8a9-9445-4917-92f5-5ca3d45d50af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:50.332879 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5c8a9-9445-4917-92f5-5ca3d45d50af None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:50.334783 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5c8a9-9445-4917-92f5-5ca3d45d50af None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:50.336591 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5c8a9-9445-4917-92f5-5ca3d45d50af None admin] PATH_INFO: `/v3/roles/member` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:50.358763 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac5c8a9-9445-4917-92f5-5ca3d45d50af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:50.365718 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-4ac5c8a9-9445-4917-92f5-5ca3d45d50af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 23:30:50.365718 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:50.365718 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:50.365718 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:50.365718 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:50.365718 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:50.365718 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:50.365718 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:50.365718 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:50.365718 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:50.365718 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:50.365718 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:50.365718 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:50.365718 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:50.365718 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:50.365718 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:50.366656 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:50.366656 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:50.366656 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:50.366656 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:50.366656 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:50.366656 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:50.366656 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:50.366656 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:50.366656 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:50.366656 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:50.366656 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:50.366656 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:50.366656 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:50.366656 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:50.366656 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:50.366656 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:50.367469 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:50.367469 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:50.367469 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:50.367469 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:50.367469 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:50.367469 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:50.367469 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:50.367469 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:50.367469 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:50.367469 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:50.367469 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 23:30:50.367469 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:30:50.367469 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 218/438] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:30:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:50.378607 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5c8a9-9445-4917-92f5-5ca3d45d50af None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:50.401524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:50.408276 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6101b-7368-458a-abe7-3ca3e60211a1 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:50.464553 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48480a-b97b-4949-83ce-3fc5f4469cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:50.465676 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48480a-b97b-4949-83ce-3fc5f4469cf1 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:50.465676 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48480a-b97b-4949-83ce-3fc5f4469cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:50.465850 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48480a-b97b-4949-83ce-3fc5f4469cf1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:50.481620 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48480a-b97b-4949-83ce-3fc5f4469cf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:50.487168 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-ab48480a-b97b-4949-83ce-3fc5f4469cf1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 23:30:50.487168 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:50.487168 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:50.487168 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:50.487168 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:50.487168 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:50.487168 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:50.487168 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:50.487168 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:50.487168 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:50.487168 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:50.487168 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:50.487168 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:50.487168 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:50.487168 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:50.487168 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:50.488134 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:50.488134 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:50.488134 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:50.488134 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:50.488134 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:50.488134 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:50.488134 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:50.488134 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:50.488134 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:50.488134 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:50.488134 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:50.488134 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:50.488134 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:50.488134 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:50.488134 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:50.488134 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:50.494145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:50.494145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:50.494145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:50.494145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:50.494145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:50.494145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:50.494145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:50.494145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:50.494145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:50.494145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:50.494145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 23:30:50.494145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:30:50.494145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 221/439] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 17 23:30:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:50.494145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48480a-b97b-4949-83ce-3fc5f4469cf1 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:50.563861 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5c8a9-9445-4917-92f5-5ca3d45d50af None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:50.576104 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:50.610613 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c7908-1fc3-4ed3-99b0-f634046bfff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:50.611228 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c7908-1fc3-4ed3-99b0-f634046bfff0 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:50.611515 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c7908-1fc3-4ed3-99b0-f634046bfff0 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:50.611794 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c7908-1fc3-4ed3-99b0-f634046bfff0 None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:50.612526 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c7908-1fc3-4ed3-99b0-f634046bfff0 None admin] RBAC: Adding query filter params (name=member) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:50.612942 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c7908-1fc3-4ed3-99b0-f634046bfff0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:50.614318 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c7908-1fc3-4ed3-99b0-f634046bfff0 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:50.622992 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 219/440] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 17 23:30:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:50.623745 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c7908-1fc3-4ed3-99b0-f634046bfff0 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:50.637649 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48480a-b97b-4949-83ce-3fc5f4469cf1 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:50.681551 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ce898-34a8-423c-858e-437551a9eee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:50.682374 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ce898-34a8-423c-858e-437551a9eee5 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:50.682780 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ce898-34a8-423c-858e-437551a9eee5 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:50.683059 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ce898-34a8-423c-858e-437551a9eee5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:50.697698 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6ce898-34a8-423c-858e-437551a9eee5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:50.703035 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-fc6ce898-34a8-423c-858e-437551a9eee5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:30:50.703035 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:50.703035 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:50.703035 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:50.703035 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:50.703035 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:50.703035 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:50.703035 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:50.703035 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:50.703035 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:50.703035 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:50.703035 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:50.703035 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:50.703035 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:50.703035 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:50.703035 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:50.704164 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:50.704164 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:50.704164 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:50.704164 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:50.704164 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:50.704164 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:50.704164 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:50.704164 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:50.704164 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:50.704164 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:50.704164 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:50.704164 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:50.704164 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:50.704164 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:50.704164 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:50.704164 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:50.705143 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:50.705143 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:50.705143 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:50.705143 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:50.705143 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:50.705143 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:50.705143 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:50.705143 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:50.705143 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:50.705143 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:50.705143 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:30:50.705143 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:30:50.705143 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 222/441] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 17 23:30:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:50.705957 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ce898-34a8-423c-858e-437551a9eee5 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:50.730837 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:50.739372 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c7908-1fc3-4ed3-99b0-f634046bfff0 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:50.775492 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7408a5c-f27c-457e-a595-c4843d911ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:50.776397 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7408a5c-f27c-457e-a595-c4843d911ddf None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:50.776681 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7408a5c-f27c-457e-a595-c4843d911ddf None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:50.777136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7408a5c-f27c-457e-a595-c4843d911ddf None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:50.778383 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7408a5c-f27c-457e-a595-c4843d911ddf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:50.778708 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7408a5c-f27c-457e-a595-c4843d911ddf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:50.780368 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7408a5c-f27c-457e-a595-c4843d911ddf None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:50.789946 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 220/442] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:30:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:50.790719 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7408a5c-f27c-457e-a595-c4843d911ddf None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:50.821986 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ce898-34a8-423c-858e-437551a9eee5 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:50.826833 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:50.862234 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54918c31-e709-4478-9fb9-4aec5634986f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:50.862849 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54918c31-e709-4478-9fb9-4aec5634986f None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:50.863131 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54918c31-e709-4478-9fb9-4aec5634986f None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:50.863404 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54918c31-e709-4478-9fb9-4aec5634986f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:50.906263 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7408a5c-f27c-457e-a595-c4843d911ddf None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:50.938611 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54918c31-e709-4478-9fb9-4aec5634986f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:50.940116 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54918c31-e709-4478-9fb9-4aec5634986f None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:50.942925 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2265df68-ad50-4a3b-8222-dbc8031f7edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:50.943536 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265df68-ad50-4a3b-8222-dbc8031f7edb None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:50.943823 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265df68-ad50-4a3b-8222-dbc8031f7edb None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:50.944147 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265df68-ad50-4a3b-8222-dbc8031f7edb None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:50.944791 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2265df68-ad50-4a3b-8222-dbc8031f7edb None admin] RBAC: Adding query filter params (name=admin) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:50.945100 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2265df68-ad50-4a3b-8222-dbc8031f7edb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:50.946451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2265df68-ad50-4a3b-8222-dbc8031f7edb None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:50.954761 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 221/443] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:30:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:50.955751 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2265df68-ad50-4a3b-8222-dbc8031f7edb None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:50.959163 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:51.053155 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 223/444] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:30:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:30:51.054764 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54918c31-e709-4478-9fb9-4aec5634986f None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:51.068453 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:51.077870 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2265df68-ad50-4a3b-8222-dbc8031f7edb None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:51.116588 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c9368-0c1c-4633-b87c-9dfa5a242033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:51.117198 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c9368-0c1c-4633-b87c-9dfa5a242033 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:51.117480 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c9368-0c1c-4633-b87c-9dfa5a242033 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:51.117754 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c9368-0c1c-4633-b87c-9dfa5a242033 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:51.176853 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54918c31-e709-4478-9fb9-4aec5634986f None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:51.190291 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300c9368-0c1c-4633-b87c-9dfa5a242033 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:51.191748 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300c9368-0c1c-4633-b87c-9dfa5a242033 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:51.198138 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:51.219124 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad624354-49be-4f08-b453-bec034fd5228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:51.219708 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad624354-49be-4f08-b453-bec034fd5228 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:51.220005 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad624354-49be-4f08-b453-bec034fd5228 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:51.220277 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad624354-49be-4f08-b453-bec034fd5228 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:51.294824 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:51.299175 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad624354-49be-4f08-b453-bec034fd5228 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:51.302397 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad624354-49be-4f08-b453-bec034fd5228 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:51.304888 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 222/445] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:30:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:30:51.305788 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c9368-0c1c-4633-b87c-9dfa5a242033 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:51.409408 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 224/446] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:30:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:30:51.410342 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad624354-49be-4f08-b453-bec034fd5228 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:51.418725 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c9368-0c1c-4633-b87c-9dfa5a242033 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:51.431002 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:51.465172 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546b584-dc70-47aa-8a6d-e3c25c0e200d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:51.465809 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546b584-dc70-47aa-8a6d-e3c25c0e200d None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:51.466085 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546b584-dc70-47aa-8a6d-e3c25c0e200d None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:51.466457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546b584-dc70-47aa-8a6d-e3c25c0e200d None admin] PATH_INFO: `/v3/projects/e959a87f43754d66b9bd795c08d2d827` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:51.467595 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2546b584-dc70-47aa-8a6d-e3c25c0e200d None admin] RBAC: Authorizing `identity:get_project(project_id=e959a87f43754d66b9bd795c08d2d827)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:51.469193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2546b584-dc70-47aa-8a6d-e3c25c0e200d None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:51.470918 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 223/447] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:51 2022] GET /identity/v3/projects/e959a87f43754d66b9bd795c08d2d827 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:51.471769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546b584-dc70-47aa-8a6d-e3c25c0e200d None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:51.536881 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad624354-49be-4f08-b453-bec034fd5228 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:51.544010 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:51.575116 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ce775-3503-415d-a5e0-fd281327e22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:51.575881 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ce775-3503-415d-a5e0-fd281327e22c None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:51.576259 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ce775-3503-415d-a5e0-fd281327e22c None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:51.576591 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ce775-3503-415d-a5e0-fd281327e22c None admin] PATH_INFO: `/v3/projects/e959a87f43754d66b9bd795c08d2d827` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:51.577786 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1ce775-3503-415d-a5e0-fd281327e22c None admin] RBAC: Authorizing `identity:get_project(project_id=e959a87f43754d66b9bd795c08d2d827)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:51.579320 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1ce775-3503-415d-a5e0-fd281327e22c None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:51.585362 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 225/448] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:51 2022] GET /identity/v3/projects/e959a87f43754d66b9bd795c08d2d827 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:51.585362 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ce775-3503-415d-a5e0-fd281327e22c None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:51.592394 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546b584-dc70-47aa-8a6d-e3c25c0e200d None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:51.629886 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ce1c1-f793-490f-b184-f773b6b683fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:51.630510 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ce1c1-f793-490f-b184-f773b6b683fc None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:51.630791 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ce1c1-f793-490f-b184-f773b6b683fc None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:51.631065 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ce1c1-f793-490f-b184-f773b6b683fc None admin] PATH_INFO: `/v3/projects/fd475051e9cd456abbb6cff649b374cc` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:51.632210 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ce1c1-f793-490f-b184-f773b6b683fc None admin] RBAC: Authorizing `identity:get_project(project_id=fd475051e9cd456abbb6cff649b374cc)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:51.633641 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ce1c1-f793-490f-b184-f773b6b683fc None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:51.635247 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 224/449] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:30:51 2022] GET /identity/v3/projects/fd475051e9cd456abbb6cff649b374cc => generated 344 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:51.636113 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ce1c1-f793-490f-b184-f773b6b683fc None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:51.673726 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:51.690312 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ce775-3503-415d-a5e0-fd281327e22c None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:51.728913 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad3300-3a81-4f4d-b291-8f09491324cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:51.729552 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad3300-3a81-4f4d-b291-8f09491324cc None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:51.730722 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad3300-3a81-4f4d-b291-8f09491324cc None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:51.731009 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad3300-3a81-4f4d-b291-8f09491324cc None admin] PATH_INFO: `/v3/groups/ac7c419a3715488f9ab314b2075879da` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:51.732218 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad3300-3a81-4f4d-b291-8f09491324cc None admin] RBAC: Authorizing `identity:get_group(group_id=ac7c419a3715488f9ab314b2075879da)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:51.733662 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad3300-3a81-4f4d-b291-8f09491324cc None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:51.736009 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 226/450] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Aug 17 23:30:51 2022] GET /identity/v3/groups/ac7c419a3715488f9ab314b2075879da => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:51.736009 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad3300-3a81-4f4d-b291-8f09491324cc None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:51.746560 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ce1c1-f793-490f-b184-f773b6b683fc None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:51.779505 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:51.787411 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abe4ff9-e00a-406b-8522-f5347eaf783a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:51.788072 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe4ff9-e00a-406b-8522-f5347eaf783a None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:51.788349 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe4ff9-e00a-406b-8522-f5347eaf783a None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:51.788631 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe4ff9-e00a-406b-8522-f5347eaf783a None admin] PATH_INFO: `/v3/groups/ac7c419a3715488f9ab314b2075879da` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:51.789745 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abe4ff9-e00a-406b-8522-f5347eaf783a None admin] RBAC: Authorizing `identity:get_group(group_id=ac7c419a3715488f9ab314b2075879da)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:51.791252 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abe4ff9-e00a-406b-8522-f5347eaf783a None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:51.793734 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 225/451] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Aug 17 23:30:51 2022] GET /identity/v3/groups/ac7c419a3715488f9ab314b2075879da => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:51.794301 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abe4ff9-e00a-406b-8522-f5347eaf783a None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:51.840783 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad3300-3a81-4f4d-b291-8f09491324cc None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:51.875552 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c183d5-de60-4703-a43f-f22bd410067c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:51.876299 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c183d5-de60-4703-a43f-f22bd410067c None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:51.876583 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c183d5-de60-4703-a43f-f22bd410067c None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:51.876858 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c183d5-de60-4703-a43f-f22bd410067c None admin] PATH_INFO: `/v3/groups/3ca1ae290f5c4c6c9fdf541e01401cf0` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:51.877986 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c183d5-de60-4703-a43f-f22bd410067c None admin] RBAC: Authorizing `identity:get_group(group_id=3ca1ae290f5c4c6c9fdf541e01401cf0)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:51.879457 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c183d5-de60-4703-a43f-f22bd410067c None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:51.881102 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 227/452] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Aug 17 23:30:51 2022] GET /identity/v3/groups/3ca1ae290f5c4c6c9fdf541e01401cf0 => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:51.881853 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c183d5-de60-4703-a43f-f22bd410067c None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:51.906372 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abe4ff9-e00a-406b-8522-f5347eaf783a None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:51.913025 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:51.945155 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b23ba-2ff9-4339-8348-b6a1538d08d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:51.945724 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b23ba-2ff9-4339-8348-b6a1538d08d7 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:51.945966 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b23ba-2ff9-4339-8348-b6a1538d08d7 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:51.946246 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b23ba-2ff9-4339-8348-b6a1538d08d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:51.946965 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b23ba-2ff9-4339-8348-b6a1538d08d7 None admin] RBAC: Adding query filter params (group.id=ac7c419a3715488f9ab314b2075879da, role.id=1182317a465349478dde4e28cb7cf96f, scope.project.id=e959a87f43754d66b9bd795c08d2d827) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:51.947280 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b23ba-2ff9-4339-8348-b6a1538d08d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:51.948754 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b23ba-2ff9-4339-8348-b6a1538d08d7 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:51.954417 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 226/453] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Aug 17 23:30:51 2022] GET /identity/v3/role_assignments?group.id=ac7c419a3715488f9ab314b2075879da&scope.project.id=e959a87f43754d66b9bd795c08d2d827&role.id=1182317a465349478dde4e28cb7cf96f => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:51.955186 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b23ba-2ff9-4339-8348-b6a1538d08d7 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:52.041830 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:52.059238 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c183d5-de60-4703-a43f-f22bd410067c None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:52.100971 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b23ba-2ff9-4339-8348-b6a1538d08d7 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:52.114105 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc69bb7-22de-4255-8b8b-78951726f586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:52.114964 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc69bb7-22de-4255-8b8b-78951726f586 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:52.115469 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc69bb7-22de-4255-8b8b-78951726f586 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:52.115953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc69bb7-22de-4255-8b8b-78951726f586 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:52.116856 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc69bb7-22de-4255-8b8b-78951726f586 None admin] RBAC: Adding query filter params (group.id=ac7c419a3715488f9ab314b2075879da, role.id=3428ff8e92274de6817b7e8ee9a56d13, scope.project.id=e959a87f43754d66b9bd795c08d2d827) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:52.118035 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc69bb7-22de-4255-8b8b-78951726f586 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:52.119482 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc69bb7-22de-4255-8b8b-78951726f586 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:52.126223 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 228/454] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Aug 17 23:30:51 2022] GET /identity/v3/role_assignments?group.id=ac7c419a3715488f9ab314b2075879da&scope.project.id=e959a87f43754d66b9bd795c08d2d827&role.id=3428ff8e92274de6817b7e8ee9a56d13 => generated 611 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:52.149824 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cde9178-d1ca-4aad-809b-e428a0d7da29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:52.150425 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde9178-d1ca-4aad-809b-e428a0d7da29 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:52.150698 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde9178-d1ca-4aad-809b-e428a0d7da29 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:52.150974 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde9178-d1ca-4aad-809b-e428a0d7da29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:52.152649 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cde9178-d1ca-4aad-809b-e428a0d7da29 None admin] RBAC: Adding query filter params (group.id=3ca1ae290f5c4c6c9fdf541e01401cf0, role.id=33bd60d3b0e94dcbbfe97e87f91a8d78, scope.project.id=fd475051e9cd456abbb6cff649b374cc) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:52.153086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cde9178-d1ca-4aad-809b-e428a0d7da29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:52.154651 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cde9178-d1ca-4aad-809b-e428a0d7da29 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:52.161572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 227/455] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Aug 17 23:30:51 2022] GET /identity/v3/role_assignments?group.id=3ca1ae290f5c4c6c9fdf541e01401cf0&scope.project.id=fd475051e9cd456abbb6cff649b374cc&role.id=33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 611 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:54.921973 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92a943-ed4f-4e7a-86e0-a648107a7706 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:54.922484 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92a943-ed4f-4e7a-86e0-a648107a7706 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:54.922560 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92a943-ed4f-4e7a-86e0-a648107a7706 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:54.924143 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 229/456] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:54.929114 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f0815-9dbb-4cb8-8fc1-65220f2546d2 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:54.929922 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f0815-9dbb-4cb8-8fc1-65220f2546d2 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:54.930390 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f0815-9dbb-4cb8-8fc1-65220f2546d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:54.952953 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-009f0815-9dbb-4cb8-8fc1-65220f2546d2 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:54.954418 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-009f0815-9dbb-4cb8-8fc1-65220f2546d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:55.020070 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9b8a4-dad8-4cfb-9265-14c678d97863 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:55.020368 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9b8a4-dad8-4cfb-9265-14c678d97863 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:55.020610 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9b8a4-dad8-4cfb-9265-14c678d97863 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:55.022279 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 230/457] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:55.028401 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d63cc-cf41-41cc-9098-f458130556f4 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:55.028758 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d63cc-cf41-41cc-9098-f458130556f4 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:55.029106 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d63cc-cf41-41cc-9098-f458130556f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:55.039582 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 228/458] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:55.040879 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269df29-2f8a-493a-ad23-c028713ec01d None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:55.041157 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269df29-2f8a-493a-ad23-c028713ec01d None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:55.041430 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269df29-2f8a-493a-ad23-c028713ec01d None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:55.042948 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 229/459] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:55.048016 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92061f0-ebc7-4591-9b26-2ebba5275816 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:55.048303 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92061f0-ebc7-4591-9b26-2ebba5275816 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:55.048578 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92061f0-ebc7-4591-9b26-2ebba5275816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:55.049706 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-cc0d63cc-cf41-41cc-9098-f458130556f4 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:55.051035 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-cc0d63cc-cf41-41cc-9098-f458130556f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:55.066379 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-e92061f0-ebc7-4591-9b26-2ebba5275816 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:55.067552 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-e92061f0-ebc7-4591-9b26-2ebba5275816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:55.120589 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:55.123878 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 231/460] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 23:30:55.140324 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 230/461] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 23:30:55.382825 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d63cc-cf41-41cc-9098-f458130556f4 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:55.399714 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:55.407273 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92061f0-ebc7-4591-9b26-2ebba5275816 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:55.524529 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d63cc-cf41-41cc-9098-f458130556f4 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:55.560894 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e28e7c1-7e67-4501-abc0-f6528d18c49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:55.561625 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e28e7c1-7e67-4501-abc0-f6528d18c49f None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:55.561906 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e28e7c1-7e67-4501-abc0-f6528d18c49f None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:55.562263 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e28e7c1-7e67-4501-abc0-f6528d18c49f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:55.568678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92061f0-ebc7-4591-9b26-2ebba5275816 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:55.572497 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e28e7c1-7e67-4501-abc0-f6528d18c49f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:55.574586 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e28e7c1-7e67-4501-abc0-f6528d18c49f None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:55.589752 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-2e28e7c1-7e67-4501-abc0-f6528d18c49f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:30:55.589752 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:55.589752 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:55.589752 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:55.589752 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:55.589752 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:55.589752 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:55.589752 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:55.589752 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:55.589752 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:55.589752 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:55.589752 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:55.589752 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:30:55.589752 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:30:55.589752 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:30:55.589752 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:30:55.590683 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:55.590683 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:55.590683 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:55.590683 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:55.590683 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:55.590683 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:55.590683 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:55.590683 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:55.590683 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:55.590683 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:55.590683 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:55.590683 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:55.590683 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:55.590683 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:55.590683 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:55.590683 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:55.590683 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:30:55.591952 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:30:55.591952 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:30:55.591952 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:30:55.591952 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 232/462] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:30:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:55.591952 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e28e7c1-7e67-4501-abc0-f6528d18c49f None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:55.608646 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5442ee3-39a1-4f87-8bea-bf4f99793b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:55.609289 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5442ee3-39a1-4f87-8bea-bf4f99793b1c None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:55.609570 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5442ee3-39a1-4f87-8bea-bf4f99793b1c None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:55.610166 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5442ee3-39a1-4f87-8bea-bf4f99793b1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:55.620097 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5442ee3-39a1-4f87-8bea-bf4f99793b1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:55.622084 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5442ee3-39a1-4f87-8bea-bf4f99793b1c None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:55.638345 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-e5442ee3-39a1-4f87-8bea-bf4f99793b1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:30:55.638345 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:55.638345 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:55.638345 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:55.638345 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:55.638345 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:55.638345 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:55.638345 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:55.638345 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:55.638345 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:55.638345 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:55.638345 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:55.638345 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:30:55.638345 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:30:55.638345 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:30:55.638345 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:30:55.639242 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:55.639242 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:55.639242 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:55.639242 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:55.639242 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:55.639242 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:55.639242 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:55.639242 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:55.639242 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:55.639242 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:55.639242 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:55.639242 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:55.639242 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:55.639242 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:55.639242 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:55.639242 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:55.639242 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:30:55.640217 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:30:55.640217 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:30:55.640217 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:30:55.640217 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 231/463] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:30:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:55.640894 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c2cbc-7624-46e8-bb18-b991c09f6068 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:55.641149 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c2cbc-7624-46e8-bb18-b991c09f6068 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:55.641417 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c2cbc-7624-46e8-bb18-b991c09f6068 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:55.646587 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 232/464] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:55.646587 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c2cbc-7624-46e8-bb18-b991c09f6068 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:55.673639 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:55.691038 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:55.714059 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e28e7c1-7e67-4501-abc0-f6528d18c49f None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:55.748623 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209843f6-36d2-4ed6-9b6f-984ed1fd136c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:55.749255 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209843f6-36d2-4ed6-9b6f-984ed1fd136c None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:55.749535 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209843f6-36d2-4ed6-9b6f-984ed1fd136c None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:55.749808 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209843f6-36d2-4ed6-9b6f-984ed1fd136c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:55.757476 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c2cbc-7624-46e8-bb18-b991c09f6068 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:55.760420 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209843f6-36d2-4ed6-9b6f-984ed1fd136c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:55.762421 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209843f6-36d2-4ed6-9b6f-984ed1fd136c None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:55.776547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-209843f6-36d2-4ed6-9b6f-984ed1fd136c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:30:55.776547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:55.776547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:55.776547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:55.776547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:55.776547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:55.776547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:55.776547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:55.776547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:55.776547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:55.776547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:55.776547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:55.776547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:30:55.776547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:30:55.776547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:30:55.776547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:30:55.778529 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:55.778529 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:55.778529 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:55.778529 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:55.778529 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:55.778529 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:55.778529 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:55.778529 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:55.778529 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:55.778529 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:55.778529 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:55.778529 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:55.778529 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:55.778529 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:55.778529 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:55.778529 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:55.778529 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:30:55.779386 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:30:55.779386 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:30:55.779386 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:30:55.779386 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 233/465] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:30:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:55.779613 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209843f6-36d2-4ed6-9b6f-984ed1fd136c None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:55.797611 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f880e17-7d91-4f65-b68f-15393a55b382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:55.798420 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f880e17-7d91-4f65-b68f-15393a55b382 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:55.798847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f880e17-7d91-4f65-b68f-15393a55b382 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:55.799119 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f880e17-7d91-4f65-b68f-15393a55b382 None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:55.799767 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f880e17-7d91-4f65-b68f-15393a55b382 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:55.800098 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f880e17-7d91-4f65-b68f-15393a55b382 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:55.801459 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f880e17-7d91-4f65-b68f-15393a55b382 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:55.814127 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 233/466] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:30:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:55.819846 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74471f-5b3d-4d30-a02a-b23960b4c65e None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:55.820159 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74471f-5b3d-4d30-a02a-b23960b4c65e None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:55.820438 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74471f-5b3d-4d30-a02a-b23960b4c65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:55.839998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-2d74471f-5b3d-4d30-a02a-b23960b4c65e None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:55.841421 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-2d74471f-5b3d-4d30-a02a-b23960b4c65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:55.881132 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209843f6-36d2-4ed6-9b6f-984ed1fd136c None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:55.898713 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:55.920076 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 234/467] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:55.920076 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74471f-5b3d-4d30-a02a-b23960b4c65e None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:55.922131 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5fe9da-503d-401d-b938-b90cc8287830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:55.922586 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fe9da-503d-401d-b938-b90cc8287830 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:55.922836 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fe9da-503d-401d-b938-b90cc8287830 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:55.923113 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fe9da-503d-401d-b938-b90cc8287830 None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:55.923765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5fe9da-503d-401d-b938-b90cc8287830 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:55.924103 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5fe9da-503d-401d-b938-b90cc8287830 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:55.925498 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5fe9da-503d-401d-b938-b90cc8287830 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:55.942871 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 234/468] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:30:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:55.943896 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5fe9da-503d-401d-b938-b90cc8287830 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:55.953124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:56.046714 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74471f-5b3d-4d30-a02a-b23960b4c65e None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:56.059194 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5fe9da-503d-401d-b938-b90cc8287830 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:56.091555 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b979f489-9ae7-4a85-b67a-19ae67b2a5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:56.092201 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979f489-9ae7-4a85-b67a-19ae67b2a5af None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:56.092872 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979f489-9ae7-4a85-b67a-19ae67b2a5af None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:56.093152 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979f489-9ae7-4a85-b67a-19ae67b2a5af None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:56.094061 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b979f489-9ae7-4a85-b67a-19ae67b2a5af None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:56.094368 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b979f489-9ae7-4a85-b67a-19ae67b2a5af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:56.096068 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b979f489-9ae7-4a85-b67a-19ae67b2a5af None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:56.098387 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e961b7b-ff95-46b9-843c-c54fb4165e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:56.099251 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e961b7b-ff95-46b9-843c-c54fb4165e90 None admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:56.099726 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e961b7b-ff95-46b9-843c-c54fb4165e90 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:56.100213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e961b7b-ff95-46b9-843c-c54fb4165e90 None admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:56.101028 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e961b7b-ff95-46b9-843c-c54fb4165e90 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:56.102730 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e961b7b-ff95-46b9-843c-c54fb4165e90 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:56.108613 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 235/469] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:30:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:56.109227 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b979f489-9ae7-4a85-b67a-19ae67b2a5af None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:56.133888 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 235/470] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Aug 17 23:30:55 2022] POST /identity/v3/users => generated 248 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:30:56.135055 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e961b7b-ff95-46b9-843c-c54fb4165e90 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:56.208109 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:56.253041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:56.266761 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b979f489-9ae7-4a85-b67a-19ae67b2a5af None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:56.284631 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e961b7b-ff95-46b9-843c-c54fb4165e90 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:56.307294 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02f847-3cf2-48d8-987a-18cad4f3cea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:56.307847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02f847-3cf2-48d8-987a-18cad4f3cea7 None admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:56.308294 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02f847-3cf2-48d8-987a-18cad4f3cea7 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:56.308577 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02f847-3cf2-48d8-987a-18cad4f3cea7 None admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:56.309176 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02f847-3cf2-48d8-987a-18cad4f3cea7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:56.310710 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02f847-3cf2-48d8-987a-18cad4f3cea7 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:56.326318 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d40b9-fdb7-4cd1-b6e1-a5d475b61663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:56.327063 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d40b9-fdb7-4cd1-b6e1-a5d475b61663 None admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:56.327346 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d40b9-fdb7-4cd1-b6e1-a5d475b61663 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:56.327619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d40b9-fdb7-4cd1-b6e1-a5d475b61663 None admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:56.328428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412d40b9-fdb7-4cd1-b6e1-a5d475b61663 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:56.330122 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412d40b9-fdb7-4cd1-b6e1-a5d475b61663 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:56.338236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 236/471] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Aug 17 23:30:56 2022] POST /identity/v3/users => generated 248 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:30:56.339919 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02f847-3cf2-48d8-987a-18cad4f3cea7 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:56.358819 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 236/472] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Aug 17 23:30:56 2022] POST /identity/v3/users => generated 246 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:30:56.360130 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb9230-a51f-4a8c-96c1-9ccf93e87722 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:56.360411 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb9230-a51f-4a8c-96c1-9ccf93e87722 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:56.360742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb9230-a51f-4a8c-96c1-9ccf93e87722 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:56.362171 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 237/473] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:56.368157 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdd865-ef9b-4863-bf30-53b024ee0bc8 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:56.368488 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdd865-ef9b-4863-bf30-53b024ee0bc8 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:56.368712 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdd865-ef9b-4863-bf30-53b024ee0bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:56.391294 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-dffdd865-ef9b-4863-bf30-53b024ee0bc8 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:56.393353 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-dffdd865-ef9b-4863-bf30-53b024ee0bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:56.519260 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02f847-3cf2-48d8-987a-18cad4f3cea7 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:56.535993 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 238/474] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 23:30:56.562939 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e0c15-7502-49cf-aba8-8e07466f0500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:56.563578 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e0c15-7502-49cf-aba8-8e07466f0500 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:56.563917 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e0c15-7502-49cf-aba8-8e07466f0500 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:56.564222 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e0c15-7502-49cf-aba8-8e07466f0500 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:56.576047 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9e0c15-7502-49cf-aba8-8e07466f0500 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:56.577606 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9e0c15-7502-49cf-aba8-8e07466f0500 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:56.580625 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:56.594887 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-2d9e0c15-7502-49cf-aba8-8e07466f0500 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:30:56.594887 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:56.594887 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:56.594887 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:56.594887 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:56.594887 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:56.594887 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:56.594887 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:56.594887 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:56.594887 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:56.594887 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:56.594887 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:56.594887 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:30:56.594887 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:30:56.594887 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:30:56.594887 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:30:56.595850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:56.595850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:56.595850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:56.595850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:56.595850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:56.595850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:56.595850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:56.595850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:56.595850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:56.595850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:56.595850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:56.595850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:56.595850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:56.595850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:56.595850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:56.595850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:56.595850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:30:56.596739 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:30:56.596739 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:30:56.596739 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:30:56.596739 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 237/475] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:30:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:56.602119 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffdd865-ef9b-4863-bf30-53b024ee0bc8 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:56.625737 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:56.710546 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffdd865-ef9b-4863-bf30-53b024ee0bc8 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:56.746994 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc24df-3ee0-4b6a-933a-98e1fe355814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:56.747601 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc24df-3ee0-4b6a-933a-98e1fe355814 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:56.747932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc24df-3ee0-4b6a-933a-98e1fe355814 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:56.748239 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc24df-3ee0-4b6a-933a-98e1fe355814 None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:56.748948 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc24df-3ee0-4b6a-933a-98e1fe355814 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:56.749256 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc24df-3ee0-4b6a-933a-98e1fe355814 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:56.750645 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc24df-3ee0-4b6a-933a-98e1fe355814 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:56.764900 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 239/476] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:30:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:56.770465 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e0c15-7502-49cf-aba8-8e07466f0500 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:56.881627 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e0c15-7502-49cf-aba8-8e07466f0500 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:56.881810 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc24df-3ee0-4b6a-933a-98e1fe355814 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:56.928818 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0febeb9-8c62-4cec-a485-876b6cb9e014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:56.929986 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0febeb9-8c62-4cec-a485-876b6cb9e014 None admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:56.930476 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0febeb9-8c62-4cec-a485-876b6cb9e014 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:56.931188 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0febeb9-8c62-4cec-a485-876b6cb9e014 None admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:56.932542 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0febeb9-8c62-4cec-a485-876b6cb9e014 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:56.934807 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0febeb9-8c62-4cec-a485-876b6cb9e014 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:56.971774 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 238/477] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Aug 17 23:30:56 2022] POST /identity/v3/users => generated 249 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:30:56.981189 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:57.004061 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc24df-3ee0-4b6a-933a-98e1fe355814 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:57.044602 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf98ef15-edce-432e-877d-8548030cd8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:57.045119 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98ef15-edce-432e-877d-8548030cd8a4 None admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:57.045400 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98ef15-edce-432e-877d-8548030cd8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:57.045668 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98ef15-edce-432e-877d-8548030cd8a4 None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:57.046353 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf98ef15-edce-432e-877d-8548030cd8a4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:57.047829 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf98ef15-edce-432e-877d-8548030cd8a4 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:57.057058 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.sql.core [None req-cf98ef15-edce-432e-877d-8548030cd8a4 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 17 23:30:57.057058 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 17 23:30:57.057058 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [parameters: {'id': '51eb616036fe406fb7569dfd0b76ee44', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 17 23:30:57.057058 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=85835) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 17 23:30:57.074422 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-cf98ef15-edce-432e-877d-8548030cd8a4 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 17 23:30:57.074422 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:57.074422 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 17 23:30:57.074422 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 17 23:30:57.074422 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 17 23:30:57.074422 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 17 23:30:57.074422 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 17 23:30:57.074422 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application result = self._query(query) Aug 17 23:30:57.074422 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 17 23:30:57.074422 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application conn.query(q) Aug 17 23:30:57.074422 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 17 23:30:57.074422 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 17 23:30:57.074422 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 17 23:30:57.074422 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application result.read() Aug 17 23:30:57.074422 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 17 23:30:57.075396 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 17 23:30:57.075396 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 17 23:30:57.075396 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 17 23:30:57.075396 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 17 23:30:57.075396 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 17 23:30:57.075396 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 17 23:30:57.075396 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 17 23:30:57.075396 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 17 23:30:57.075396 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:30:57.075396 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 17 23:30:57.075396 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:30:57.075396 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:57.075396 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 17 23:30:57.075396 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 17 23:30:57.075396 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 17 23:30:57.075396 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return ref.to_dict() Aug 17 23:30:57.075396 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 17 23:30:57.075396 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application next(self.gen) Aug 17 23:30:57.076359 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 17 23:30:57.076359 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application yield resource Aug 17 23:30:57.076359 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 17 23:30:57.076359 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application next(self.gen) Aug 17 23:30:57.076359 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 17 23:30:57.076359 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self.session.rollback() Aug 17 23:30:57.076359 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 17 23:30:57.076359 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self.force_reraise() Aug 17 23:30:57.076359 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 17 23:30:57.076359 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise self.value Aug 17 23:30:57.076359 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 17 23:30:57.076359 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 17 23:30:57.076359 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 17 23:30:57.076359 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application session.commit() Aug 17 23:30:57.076359 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 17 23:30:57.076359 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 17 23:30:57.076359 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 17 23:30:57.077227 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self._prepare_impl() Aug 17 23:30:57.077227 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 17 23:30:57.077227 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self.session.flush() Aug 17 23:30:57.077227 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 17 23:30:57.077227 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self._flush(objects) Aug 17 23:30:57.077227 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 17 23:30:57.077227 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 17 23:30:57.077227 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 17 23:30:57.077227 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application compat.raise_( Aug 17 23:30:57.077227 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 17 23:30:57.077227 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception Aug 17 23:30:57.077227 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 17 23:30:57.077227 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application flush_context.execute() Aug 17 23:30:57.077227 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 17 23:30:57.077227 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rec.execute(self) Aug 17 23:30:57.077227 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 17 23:30:57.077227 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 17 23:30:57.078152 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 17 23:30:57.078152 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 17 23:30:57.078152 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 17 23:30:57.078152 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 17 23:30:57.078152 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 17 23:30:57.078152 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 17 23:30:57.078152 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 17 23:30:57.078152 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 17 23:30:57.078152 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 17 23:30:57.078152 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 17 23:30:57.078152 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 17 23:30:57.078152 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 17 23:30:57.078152 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 17 23:30:57.078152 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 17 23:30:57.078152 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 17 23:30:57.078152 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception Aug 17 23:30:57.078980 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 17 23:30:57.078980 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 17 23:30:57.078980 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 17 23:30:57.078980 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 17 23:30:57.078980 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 17 23:30:57.078980 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application result = self._query(query) Aug 17 23:30:57.078980 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 17 23:30:57.078980 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application conn.query(q) Aug 17 23:30:57.078980 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 17 23:30:57.078980 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 17 23:30:57.078980 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 17 23:30:57.078980 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application result.read() Aug 17 23:30:57.078980 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 17 23:30:57.078980 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 17 23:30:57.078980 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 17 23:30:57.078980 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 17 23:30:57.081962 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 17 23:30:57.081962 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 17 23:30:57.081962 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 17 23:30:57.081962 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 17 23:30:57.081962 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 17 23:30:57.081962 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 17 23:30:57.081962 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [parameters: {'id': '51eb616036fe406fb7569dfd0b76ee44', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 17 23:30:57.081962 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 17 23:30:57.081962 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:30:57.081962 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 23:30:57.081962 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:30:57.081962 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:57.081962 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:57.081962 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:57.081962 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:57.081962 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:57.086898 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:57.086898 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:57.086898 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:57.086898 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:57.086898 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:57.086898 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:57.086898 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 17 23:30:57.086898 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 17 23:30:57.086898 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:57.086898 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:57.086898 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 17 23:30:57.086898 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 17 23:30:57.086898 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 17 23:30:57.086898 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 17 23:30:57.086898 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 17 23:30:57.086898 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:30:57.090604 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 240/478] 10.222.0.26 () {60 vars in 1131 bytes} [Wed Aug 17 23:30:56 2022] POST /identity/v3/roles => generated 144 bytes in 195 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:30:57.090754 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0febeb9-8c62-4cec-a485-876b6cb9e014 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:57.178853 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:57.233310 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0febeb9-8c62-4cec-a485-876b6cb9e014 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:57.270081 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5074e9-bc6b-4c29-b942-a5007994f048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:57.270714 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5074e9-bc6b-4c29-b942-a5007994f048 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:57.270997 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5074e9-bc6b-4c29-b942-a5007994f048 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:57.271273 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5074e9-bc6b-4c29-b942-a5007994f048 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:57.282534 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5074e9-bc6b-4c29-b942-a5007994f048 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:57.287911 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-5a5074e9-bc6b-4c29-b942-a5007994f048 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 23:30:57.287911 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:57.287911 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:57.287911 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:57.287911 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:57.287911 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:57.287911 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:57.287911 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:57.287911 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:57.287911 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:57.287911 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:57.287911 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:57.287911 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:57.287911 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:57.287911 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:57.287911 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:57.288827 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:57.288827 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:57.288827 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:57.288827 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:57.288827 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:57.288827 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:57.288827 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:57.288827 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:57.288827 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:57.288827 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:57.288827 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:57.288827 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:57.288827 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:57.288827 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:57.288827 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:57.288827 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:57.289608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:57.289608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:57.289608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:57.289608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:57.289608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:57.289608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:57.289608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:57.289608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:57.289608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:57.289608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:57.289608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 23:30:57.289608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:30:57.289608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 239/479] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 17 23:30:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:57.296253 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf98ef15-edce-432e-877d-8548030cd8a4 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:57.401430 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf98ef15-edce-432e-877d-8548030cd8a4 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:57.439024 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea62be-14fe-4588-adc6-850fd388d62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:57.439647 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea62be-14fe-4588-adc6-850fd388d62c None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:57.442598 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea62be-14fe-4588-adc6-850fd388d62c None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:57.442598 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea62be-14fe-4588-adc6-850fd388d62c None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:57.442598 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea62be-14fe-4588-adc6-850fd388d62c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:57.442598 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea62be-14fe-4588-adc6-850fd388d62c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:57.442598 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea62be-14fe-4588-adc6-850fd388d62c None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:57.459176 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 241/480] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:30:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:58.597290 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf8de8-9174-49b0-a17c-009a562530b1 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:58.598292 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf8de8-9174-49b0-a17c-009a562530b1 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:58.598767 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf8de8-9174-49b0-a17c-009a562530b1 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:58.600545 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 240/481] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:58.606236 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbfd25f-502f-4932-9ab9-7e23d896f0d4 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:58.606633 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbfd25f-502f-4932-9ab9-7e23d896f0d4 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:58.606820 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbfd25f-502f-4932-9ab9-7e23d896f0d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:58.621470 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:58.629939 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-0fbfd25f-502f-4932-9ab9-7e23d896f0d4 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:58.631366 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-0fbfd25f-502f-4932-9ab9-7e23d896f0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:58.679629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a85e5e-3e6e-49cc-9448-fa4808043868 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:58.679949 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a85e5e-3e6e-49cc-9448-fa4808043868 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:58.680202 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a85e5e-3e6e-49cc-9448-fa4808043868 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:58.681654 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 241/482] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 23:30:58.687377 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9192c03-9326-48b0-aa26-836eb63f13fc None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:58.687901 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9192c03-9326-48b0-aa26-836eb63f13fc None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:58.689937 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9192c03-9326-48b0-aa26-836eb63f13fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:58.721638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-b9192c03-9326-48b0-aa26-836eb63f13fc None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:58.723160 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-b9192c03-9326-48b0-aa26-836eb63f13fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:58.737591 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 242/483] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:58.818470 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 242/484] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:59.074072 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7fa57-8664-4af2-89a7-ce2cf063b922 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:59.074713 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7fa57-8664-4af2-89a7-ce2cf063b922 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:59.075199 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7fa57-8664-4af2-89a7-ce2cf063b922 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:59.077023 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 243/485] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:59.081871 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee28e0-c46f-4bbc-beeb-e4966b127639 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:59.082387 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee28e0-c46f-4bbc-beeb-e4966b127639 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:59.082847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee28e0-c46f-4bbc-beeb-e4966b127639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:59.107301 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-51ee28e0-c46f-4bbc-beeb-e4966b127639 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:59.108809 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-51ee28e0-c46f-4bbc-beeb-e4966b127639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:59.151557 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:59.186614 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7fa57-8664-4af2-89a7-ce2cf063b922 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:59.197141 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 243/486] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:30:59.198090 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee28e0-c46f-4bbc-beeb-e4966b127639 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:59.305512 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee28e0-c46f-4bbc-beeb-e4966b127639 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:59.343136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe817703-01dd-42d8-85cb-218d7440ef77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:59.343685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe817703-01dd-42d8-85cb-218d7440ef77 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:59.343986 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe817703-01dd-42d8-85cb-218d7440ef77 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:59.344398 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe817703-01dd-42d8-85cb-218d7440ef77 None admin] PATH_INFO: `/v3/roles/service` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:59.355723 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe817703-01dd-42d8-85cb-218d7440ef77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:59.360744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-fe817703-01dd-42d8-85cb-218d7440ef77 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:30:59.360744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:59.360744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:59.360744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:59.360744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:59.360744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:59.360744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:59.360744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:59.360744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:59.360744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:59.360744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:59.360744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:59.360744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:59.360744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:59.360744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:59.360744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:59.361688 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:59.361688 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:59.361688 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:59.361688 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:59.361688 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:59.361688 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:59.361688 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:59.361688 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:59.361688 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:59.361688 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:59.361688 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:59.361688 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:59.361688 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:59.361688 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:59.361688 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:59.361688 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:59.362504 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:59.362504 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:59.362504 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:59.362504 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:59.362504 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:59.362504 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:59.362504 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:59.362504 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:59.362504 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:59.362504 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:59.362504 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:30:59.362504 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:30:59.362504 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 244/487] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 17 23:30:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:59.371312 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe817703-01dd-42d8-85cb-218d7440ef77 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:59.410242 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7fa57-8664-4af2-89a7-ce2cf063b922 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:59.448555 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee9ff61-b5e2-4e67-b40a-f05330f6070c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:59.449708 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9ff61-b5e2-4e67-b40a-f05330f6070c None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:59.450159 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9ff61-b5e2-4e67-b40a-f05330f6070c None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:59.450427 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9ff61-b5e2-4e67-b40a-f05330f6070c None admin] PATH_INFO: `/v3/roles/service` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:59.453734 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:59.463346 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee9ff61-b5e2-4e67-b40a-f05330f6070c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:59.464385 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:59.468754 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-4ee9ff61-b5e2-4e67-b40a-f05330f6070c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:30:59.468754 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:59.468754 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:59.468754 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:59.468754 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:59.468754 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:59.468754 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:59.468754 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:59.468754 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:59.468754 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:59.468754 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:59.468754 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:59.468754 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:59.468754 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:59.468754 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:59.468754 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:59.469658 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:59.469658 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:59.469658 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:59.469658 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:59.469658 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:59.469658 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:59.469658 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:59.469658 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:59.469658 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:59.469658 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:59.469658 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:59.469658 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:59.469658 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:59.469658 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:59.469658 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:59.469658 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:59.470519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:59.470519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:59.470519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:59.470519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:59.470519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:59.470519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:59.470519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:59.470519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:59.470519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:59.470519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:59.470519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:30:59.470519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:30:59.470519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 244/488] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 17 23:30:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:59.475817 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee9ff61-b5e2-4e67-b40a-f05330f6070c None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:59.534403 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe817703-01dd-42d8-85cb-218d7440ef77 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:59.569766 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34a4d22-8473-48ca-b074-51da6d7b7ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:59.570374 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a4d22-8473-48ca-b074-51da6d7b7ec9 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:59.570654 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a4d22-8473-48ca-b074-51da6d7b7ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:59.570927 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a4d22-8473-48ca-b074-51da6d7b7ec9 None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:59.571598 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34a4d22-8473-48ca-b074-51da6d7b7ec9 None admin] RBAC: Adding query filter params (name=service) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:59.571903 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34a4d22-8473-48ca-b074-51da6d7b7ec9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:59.573311 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34a4d22-8473-48ca-b074-51da6d7b7ec9 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:59.586200 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 245/489] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 17 23:30:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:59.587080 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34a4d22-8473-48ca-b074-51da6d7b7ec9 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:59.597656 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee9ff61-b5e2-4e67-b40a-f05330f6070c None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:59.635314 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b938e6f-934a-4b56-bea1-03c06855bdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:59.636395 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b938e6f-934a-4b56-bea1-03c06855bdd3 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:59.636395 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b938e6f-934a-4b56-bea1-03c06855bdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:59.636594 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b938e6f-934a-4b56-bea1-03c06855bdd3 None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:59.637173 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b938e6f-934a-4b56-bea1-03c06855bdd3 None admin] RBAC: Adding query filter params (name=service) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:59.637580 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b938e6f-934a-4b56-bea1-03c06855bdd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:59.638857 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b938e6f-934a-4b56-bea1-03c06855bdd3 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:59.647672 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 245/490] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 17 23:30:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:59.652978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b938e6f-934a-4b56-bea1-03c06855bdd3 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:59.687325 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:59.695613 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34a4d22-8473-48ca-b074-51da6d7b7ec9 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:59.716764 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:59.732884 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce65f94-262f-43da-8cc8-0f67d47b6b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:59.733514 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce65f94-262f-43da-8cc8-0f67d47b6b0a None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:59.733791 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce65f94-262f-43da-8cc8-0f67d47b6b0a None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:59.734061 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce65f94-262f-43da-8cc8-0f67d47b6b0a None admin] PATH_INFO: `/v3/roles/service` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:59.744428 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce65f94-262f-43da-8cc8-0f67d47b6b0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:30:59.750201 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-4ce65f94-262f-43da-8cc8-0f67d47b6b0a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:30:59.750201 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:59.750201 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:59.750201 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:59.750201 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:59.750201 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:59.750201 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:59.750201 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:59.750201 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:59.750201 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:59.750201 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:59.750201 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:59.750201 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:30:59.750201 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:30:59.750201 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:30:59.750201 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:30:59.751086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:30:59.751086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:30:59.751086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:30:59.751086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:30:59.751086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:59.751086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:59.751086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:59.751086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:59.751086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:59.751086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:59.751086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:59.751086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:59.751086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:59.751086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:59.751086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:59.751086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:59.751896 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:59.751896 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:59.751896 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:59.751896 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:59.751896 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:30:59.751896 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:30:59.751896 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:30:59.751896 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:30:59.751896 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:30:59.751896 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:30:59.751896 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:30:59.751896 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:30:59.751896 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 246/491] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 17 23:30:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:59.752987 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84384e5-7593-485a-b37f-4de2b6eab0d8 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:59.752987 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84384e5-7593-485a-b37f-4de2b6eab0d8 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:59.752987 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84384e5-7593-485a-b37f-4de2b6eab0d8 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:59.758097 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 247/492] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:30:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:30:59.758721 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84384e5-7593-485a-b37f-4de2b6eab0d8 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:59.767650 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b938e6f-934a-4b56-bea1-03c06855bdd3 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:59.805334 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e48bd-1d13-4504-a90b-46263817a430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:59.805933 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e48bd-1d13-4504-a90b-46263817a430 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:59.806204 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e48bd-1d13-4504-a90b-46263817a430 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:59.806463 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e48bd-1d13-4504-a90b-46263817a430 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:59.818207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6e48bd-1d13-4504-a90b-46263817a430 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:59.819638 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6e48bd-1d13-4504-a90b-46263817a430 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:59.833774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-6c6e48bd-1d13-4504-a90b-46263817a430 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:30:59.833774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:59.833774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:59.833774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:59.833774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:59.833774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:59.833774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:59.833774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:59.833774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:59.833774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:59.833774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:59.833774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:59.833774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:30:59.833774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:30:59.833774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:30:59.833774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:30:59.834717 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:59.834717 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:59.834717 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:59.834717 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:59.834717 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:59.834717 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:59.834717 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:59.834717 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:59.834717 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:59.834717 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:59.834717 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:59.834717 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:59.834717 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:59.834717 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:59.834717 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:59.834717 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:59.834717 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:30:59.835542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:30:59.835542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:30:59.835542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:30:59.835542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 246/493] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:30:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:59.835762 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e48bd-1d13-4504-a90b-46263817a430 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:30:59.870331 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84384e5-7593-485a-b37f-4de2b6eab0d8 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:59.906113 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da1ba31-c5f6-4925-892d-38caf2e26f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:59.906895 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1ba31-c5f6-4925-892d-38caf2e26f0d None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:59.907177 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1ba31-c5f6-4925-892d-38caf2e26f0d None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:59.907450 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1ba31-c5f6-4925-892d-38caf2e26f0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:59.918252 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da1ba31-c5f6-4925-892d-38caf2e26f0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:59.919705 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da1ba31-c5f6-4925-892d-38caf2e26f0d None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:59.933418 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-1da1ba31-c5f6-4925-892d-38caf2e26f0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:30:59.933418 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:30:59.933418 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:30:59.933418 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:30:59.933418 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:30:59.933418 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:30:59.933418 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:30:59.933418 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:30:59.933418 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:30:59.933418 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:30:59.933418 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:30:59.933418 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:30:59.933418 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:30:59.933418 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:30:59.933418 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:30:59.933418 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:30:59.934327 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:30:59.934327 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:30:59.934327 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:30:59.934327 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:30:59.934327 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:30:59.934327 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:30:59.934327 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:30:59.934327 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:30:59.934327 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:30:59.934327 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:30:59.934327 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:30:59.934327 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:30:59.934327 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:30:59.934327 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:30:59.934327 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:30:59.934327 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:30:59.934327 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:30:59.935368 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:30:59.935368 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:30:59.935368 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:30:59.935368 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 248/494] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:30:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:30:59.935772 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f822e30-512f-4c64-bec3-c0fb43ae5e10 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:59.936068 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f822e30-512f-4c64-bec3-c0fb43ae5e10 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:59.936343 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f822e30-512f-4c64-bec3-c0fb43ae5e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:59.941218 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e48bd-1d13-4504-a90b-46263817a430 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:30:59.948297 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:59.953856 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-3f822e30-512f-4c64-bec3-c0fb43ae5e10 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:30:59.955063 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-3f822e30-512f-4c64-bec3-c0fb43ae5e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:30:59.955640 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:30:59.979572 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b170a30-fb26-4e3b-969f-e410b5b49627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:30:59.980794 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b170a30-fb26-4e3b-969f-e410b5b49627 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:30:59.981113 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b170a30-fb26-4e3b-969f-e410b5b49627 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:30:59.981384 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b170a30-fb26-4e3b-969f-e410b5b49627 None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:30:59.982044 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b170a30-fb26-4e3b-969f-e410b5b49627 None admin] RBAC: Adding query filter params (name=service) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:30:59.982342 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b170a30-fb26-4e3b-969f-e410b5b49627 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:30:59.983721 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b170a30-fb26-4e3b-969f-e410b5b49627 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:30:59.991330 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 247/495] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 17 23:30:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:30:59.992115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b170a30-fb26-4e3b-969f-e410b5b49627 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:00.035989 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 249/496] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:30:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:31:00.035989 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39064f1-260a-4b59-b06f-97a7870f6f29 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:00.035989 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39064f1-260a-4b59-b06f-97a7870f6f29 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:00.035989 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39064f1-260a-4b59-b06f-97a7870f6f29 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:00.035989 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 250/497] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:00.035989 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39064f1-260a-4b59-b06f-97a7870f6f29 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:00.091426 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b170a30-fb26-4e3b-969f-e410b5b49627 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:00.124506 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9340a688-4363-433d-9aa1-6ec6f018800c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:00.125035 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340a688-4363-433d-9aa1-6ec6f018800c None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:00.125316 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340a688-4363-433d-9aa1-6ec6f018800c None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:00.125588 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340a688-4363-433d-9aa1-6ec6f018800c None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:00.126240 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9340a688-4363-433d-9aa1-6ec6f018800c None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:00.126544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9340a688-4363-433d-9aa1-6ec6f018800c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:00.127922 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9340a688-4363-433d-9aa1-6ec6f018800c None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:00.140608 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 248/498] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:00.141396 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9340a688-4363-433d-9aa1-6ec6f018800c None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:00.151558 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39064f1-260a-4b59-b06f-97a7870f6f29 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:00.173511 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:00.190531 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c5f422-33d5-4547-b97d-26a8c148837b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:00.191385 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c5f422-33d5-4547-b97d-26a8c148837b None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:00.191797 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c5f422-33d5-4547-b97d-26a8c148837b None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:00.192228 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c5f422-33d5-4547-b97d-26a8c148837b None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:00.192995 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c5f422-33d5-4547-b97d-26a8c148837b None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:00.193440 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c5f422-33d5-4547-b97d-26a8c148837b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:00.194820 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c5f422-33d5-4547-b97d-26a8c148837b None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:00.197093 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:00.208496 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 251/499] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:00.209820 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455b15c-82c5-4cf0-8419-b237fa948e18 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:00.210228 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455b15c-82c5-4cf0-8419-b237fa948e18 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:00.210619 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455b15c-82c5-4cf0-8419-b237fa948e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:00.228578 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-5455b15c-82c5-4cf0-8419-b237fa948e18 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:00.230544 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-5455b15c-82c5-4cf0-8419-b237fa948e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:00.251115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9340a688-4363-433d-9aa1-6ec6f018800c None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:00.288061 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa5a4a-1afe-4c31-b925-3b820ada1f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:00.288673 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa5a4a-1afe-4c31-b925-3b820ada1f73 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:00.288949 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa5a4a-1afe-4c31-b925-3b820ada1f73 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:00.289216 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa5a4a-1afe-4c31-b925-3b820ada1f73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:00.296823 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 252/500] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:31:00.297644 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5455b15c-82c5-4cf0-8419-b237fa948e18 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:00.300120 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa5a4a-1afe-4c31-b925-3b820ada1f73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:00.301691 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa5a4a-1afe-4c31-b925-3b820ada1f73 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:00.317433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-24fa5a4a-1afe-4c31-b925-3b820ada1f73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:00.317433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:00.317433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:00.317433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:00.317433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:00.317433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:00.317433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:00.317433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:00.317433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:00.317433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:00.317433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:00.317433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:00.317433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:00.317433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:00.317433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:00.317433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:00.318391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:00.318391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:00.318391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:00.318391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:00.318391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:00.318391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:00.318391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:00.318391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:00.318391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:00.318391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:00.318391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:00.318391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:00.318391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:00.318391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:00.318391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:00.318391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:00.318391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:00.319229 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:00.319229 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:00.319229 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:00.319229 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 249/501] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:00.319446 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa5a4a-1afe-4c31-b925-3b820ada1f73 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:00.402129 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5455b15c-82c5-4cf0-8419-b237fa948e18 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:00.406136 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:00.423129 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa5a4a-1afe-4c31-b925-3b820ada1f73 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:00.431110 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:00.445134 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14fbd80-faa0-4600-b9d7-71a2c41500d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:00.445877 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14fbd80-faa0-4600-b9d7-71a2c41500d3 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:00.446220 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14fbd80-faa0-4600-b9d7-71a2c41500d3 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:00.446564 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14fbd80-faa0-4600-b9d7-71a2c41500d3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:00.455841 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14fbd80-faa0-4600-b9d7-71a2c41500d3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:00.457793 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14fbd80-faa0-4600-b9d7-71a2c41500d3 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:00.461444 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb0b87a-9e20-4e46-ac60-9eae6f91a1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:00.462165 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0b87a-9e20-4e46-ac60-9eae6f91a1f1 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:00.462555 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0b87a-9e20-4e46-ac60-9eae6f91a1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:00.462918 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0b87a-9e20-4e46-ac60-9eae6f91a1f1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:00.470518 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb0b87a-9e20-4e46-ac60-9eae6f91a1f1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:00.470647 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-a14fbd80-faa0-4600-b9d7-71a2c41500d3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 17 23:31:00.470647 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:00.470647 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:00.470647 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:00.470647 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:00.470647 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:00.470647 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:00.470647 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:00.470647 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:00.470647 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:00.470647 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:00.470647 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:00.470647 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 23:31:00.470647 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 23:31:00.470647 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 23:31:00.470647 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 23:31:00.471471 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:00.471471 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:00.471471 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 23:31:00.471471 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:00.471471 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 23:31:00.471471 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:00.471471 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:00.471471 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:00.471471 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:00.471471 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:00.471471 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:00.471471 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:00.471471 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:00.471471 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:00.471471 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:00.471471 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:00.471471 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:00.471471 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:00.472417 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:00.472417 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:00.472417 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 23:31:00.472417 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 23:31:00.472417 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 23:31:00.472417 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 23:31:00.472417 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 23:31:00.472417 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 23:31:00.472417 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 17 23:31:00.472417 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:00.472417 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 253/502] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:31:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:00.472975 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14fbd80-faa0-4600-b9d7-71a2c41500d3 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:00.476466 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb0b87a-9e20-4e46-ac60-9eae6f91a1f1 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:00.490329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-cfb0b87a-9e20-4e46-ac60-9eae6f91a1f1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 17 23:31:00.490329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:00.490329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:00.490329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:00.490329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:00.490329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:00.490329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:00.490329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:00.490329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:00.490329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:00.490329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:00.490329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:00.490329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 23:31:00.490329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 23:31:00.490329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 23:31:00.490329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 23:31:00.496724 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:00.496724 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:00.496724 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 23:31:00.496724 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:00.496724 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 23:31:00.496724 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:00.496724 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:00.496724 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:00.496724 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:00.496724 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:00.496724 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:00.496724 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:00.496724 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:00.496724 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:00.496724 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:00.496724 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:00.496724 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:00.496724 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:00.497887 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:00.497887 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:00.497887 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 23:31:00.497887 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 23:31:00.497887 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 23:31:00.497887 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 23:31:00.497887 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 23:31:00.497887 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 23:31:00.497887 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 17 23:31:00.497887 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:00.497887 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 250/503] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:31:00 2022] GET /identity/v3/users/glance => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:00.497887 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb0b87a-9e20-4e46-ac60-9eae6f91a1f1 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:00.587117 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14fbd80-faa0-4600-b9d7-71a2c41500d3 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:00.599919 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb0b87a-9e20-4e46-ac60-9eae6f91a1f1 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:00.620009 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b6194-1dfd-4697-8330-f28250618eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:00.621224 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b6194-1dfd-4697-8330-f28250618eb6 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:00.621508 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b6194-1dfd-4697-8330-f28250618eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:00.621784 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b6194-1dfd-4697-8330-f28250618eb6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:00.632044 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163b6194-1dfd-4697-8330-f28250618eb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:31:00.633324 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964e329-ddc3-4536-90c8-19cc21f7899b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:00.633923 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964e329-ddc3-4536-90c8-19cc21f7899b None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:00.634205 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964e329-ddc3-4536-90c8-19cc21f7899b None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:00.634476 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964e329-ddc3-4536-90c8-19cc21f7899b None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:00.635137 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0964e329-ddc3-4536-90c8-19cc21f7899b None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:00.635422 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0964e329-ddc3-4536-90c8-19cc21f7899b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:00.636906 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0964e329-ddc3-4536-90c8-19cc21f7899b None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:00.637271 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-163b6194-1dfd-4697-8330-f28250618eb6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:31:00.637271 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:00.637271 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:00.637271 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:00.637271 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:00.637271 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:00.637271 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:00.637271 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:00.637271 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:00.637271 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:00.637271 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:00.637271 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:00.637271 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:31:00.637271 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:31:00.637271 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:31:00.637271 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:31:00.638268 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:31:00.638268 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:31:00.638268 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:31:00.638268 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:31:00.638268 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:00.638268 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:00.638268 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:00.638268 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:00.638268 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:00.638268 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:00.638268 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:00.638268 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:00.638268 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:00.638268 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:00.638268 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:00.638268 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:00.639246 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:00.639246 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:00.639246 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:00.639246 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:00.639246 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:31:00.639246 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:31:00.639246 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:31:00.639246 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:31:00.639246 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:31:00.639246 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:31:00.639246 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:31:00.639246 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:00.639246 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 254/504] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 17 23:31:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:00.639246 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b6194-1dfd-4697-8330-f28250618eb6 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:00.651019 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 251/505] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:00.651019 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964e329-ddc3-4536-90c8-19cc21f7899b None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:00.656863 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:00.686716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:00.741735 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b6194-1dfd-4697-8330-f28250618eb6 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:00.749718 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964e329-ddc3-4536-90c8-19cc21f7899b None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:00.773752 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135d769-6b58-4cf3-be3d-27d7575cc204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:00.774353 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135d769-6b58-4cf3-be3d-27d7575cc204 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:00.774633 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135d769-6b58-4cf3-be3d-27d7575cc204 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:00.774908 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135d769-6b58-4cf3-be3d-27d7575cc204 None admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:00.775633 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3135d769-6b58-4cf3-be3d-27d7575cc204 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:00.775940 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3135d769-6b58-4cf3-be3d-27d7575cc204 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:00.778188 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3135d769-6b58-4cf3-be3d-27d7575cc204 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:00.783131 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab15a2-7f93-43c6-9c59-e9e3a30b4fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:00.783779 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab15a2-7f93-43c6-9c59-e9e3a30b4fcb None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:00.784100 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab15a2-7f93-43c6-9c59-e9e3a30b4fcb None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:00.784374 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab15a2-7f93-43c6-9c59-e9e3a30b4fcb None admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:00.785089 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab15a2-7f93-43c6-9c59-e9e3a30b4fcb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:00.785392 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab15a2-7f93-43c6-9c59-e9e3a30b4fcb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:00.786819 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab15a2-7f93-43c6-9c59-e9e3a30b4fcb None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:00.822726 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 252/506] 10.222.0.26 () {56 vars in 1137 bytes} [Wed Aug 17 23:31:00 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:00.823566 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab15a2-7f93-43c6-9c59-e9e3a30b4fcb None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:00.833893 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 255/507] 10.222.0.26 () {56 vars in 1137 bytes} [Wed Aug 17 23:31:00 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:00.833893 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135d769-6b58-4cf3-be3d-27d7575cc204 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:00.885437 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:00.924414 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab15a2-7f93-43c6-9c59-e9e3a30b4fcb None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:00.931616 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135d769-6b58-4cf3-be3d-27d7575cc204 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:00.933878 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:00.957034 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e8c49-2564-4907-b5d6-533e2192b90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:00.957656 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e8c49-2564-4907-b5d6-533e2192b90f None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:00.957932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e8c49-2564-4907-b5d6-533e2192b90f None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:00.958203 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e8c49-2564-4907-b5d6-533e2192b90f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:00.965770 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec28b385-1755-4cf0-ac16-157eae9a341a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:00.966427 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec28b385-1755-4cf0-ac16-157eae9a341a None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:00.966708 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec28b385-1755-4cf0-ac16-157eae9a341a None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:00.966979 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec28b385-1755-4cf0-ac16-157eae9a341a None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:00.967643 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec28b385-1755-4cf0-ac16-157eae9a341a None admin] RBAC: Adding query filter params (name=service) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:00.967943 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec28b385-1755-4cf0-ac16-157eae9a341a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:00.968578 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154e8c49-2564-4907-b5d6-533e2192b90f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:00.969349 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec28b385-1755-4cf0-ac16-157eae9a341a None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:00.970019 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154e8c49-2564-4907-b5d6-533e2192b90f None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:00.977620 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 256/508] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 17 23:31:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:00.978545 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec28b385-1755-4cf0-ac16-157eae9a341a None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:00.989027 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-154e8c49-2564-4907-b5d6-533e2192b90f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:00.989027 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:00.989027 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:00.989027 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:00.989027 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:00.989027 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:00.989027 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:00.989027 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:00.989027 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:00.989027 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:00.989027 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:00.989027 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:00.989027 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:00.989027 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:00.989027 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:00.989027 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:00.990115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:00.990115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:00.990115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:00.990115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:00.990115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:00.990115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:00.990115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:00.990115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:00.990115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:00.990115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:00.990115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:00.990115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:00.990115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:00.990115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:00.990115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:00.990115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:00.990115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:00.991013 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:00.991013 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:00.991013 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:00.991013 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 253/509] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:00.991243 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e8c49-2564-4907-b5d6-533e2192b90f None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:01.077926 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec28b385-1755-4cf0-ac16-157eae9a341a None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:01.091183 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e8c49-2564-4907-b5d6-533e2192b90f None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:01.104788 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:01.112423 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021bfbb-559a-44ca-a365-aaf2a470a93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:01.113041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021bfbb-559a-44ca-a365-aaf2a470a93a None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:01.113317 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021bfbb-559a-44ca-a365-aaf2a470a93a None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:01.113590 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021bfbb-559a-44ca-a365-aaf2a470a93a None admin] PATH_INFO: `/v3/users/nova` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:01.121142 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d021bfbb-559a-44ca-a365-aaf2a470a93a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:01.122668 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d021bfbb-559a-44ca-a365-aaf2a470a93a None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:01.124938 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d146b2-f581-45c9-b584-8e85bcfd0f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:01.125527 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d146b2-f581-45c9-b584-8e85bcfd0f88 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:01.125798 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d146b2-f581-45c9-b584-8e85bcfd0f88 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:01.126064 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d146b2-f581-45c9-b584-8e85bcfd0f88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:01.134894 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-d021bfbb-559a-44ca-a365-aaf2a470a93a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 17 23:31:01.134894 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:01.134894 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:01.134894 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:01.134894 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:01.134894 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:01.134894 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:01.134894 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:01.134894 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:01.134894 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:01.134894 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:01.134894 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:01.134894 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 23:31:01.134894 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 23:31:01.134894 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 23:31:01.134894 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 23:31:01.136484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:01.136484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:01.136484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 23:31:01.136484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:01.136484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 23:31:01.136484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:01.136484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:01.136484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:01.136484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:01.136484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:01.136484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:01.136484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:01.136484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:01.136484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:01.136484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:01.136484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:01.136484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:01.136484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:01.137387 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:01.137387 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:01.137387 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 23:31:01.137387 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 23:31:01.137387 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 23:31:01.137387 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 23:31:01.137387 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 23:31:01.137387 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 23:31:01.137387 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 17 23:31:01.137387 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:01.137387 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 257/510] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Aug 17 23:31:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:01.138308 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021bfbb-559a-44ca-a365-aaf2a470a93a None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:01.165987 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:01.199690 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d146b2-f581-45c9-b584-8e85bcfd0f88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:01.201115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d146b2-f581-45c9-b584-8e85bcfd0f88 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:01.236111 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021bfbb-559a-44ca-a365-aaf2a470a93a None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:01.268910 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5052687d-f4d8-4ab2-b81f-2cd38c740350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:01.269501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052687d-f4d8-4ab2-b81f-2cd38c740350 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:01.269775 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052687d-f4d8-4ab2-b81f-2cd38c740350 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:01.270043 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052687d-f4d8-4ab2-b81f-2cd38c740350 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:01.301332 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 254/511] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:31:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:31:01.301332 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d146b2-f581-45c9-b584-8e85bcfd0f88 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:01.311453 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:01.340885 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5052687d-f4d8-4ab2-b81f-2cd38c740350 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:01.342386 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5052687d-f4d8-4ab2-b81f-2cd38c740350 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:01.366625 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:01.395220 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d146b2-f581-45c9-b584-8e85bcfd0f88 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:01.429323 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2817849-b374-4498-9103-f5ffb9806074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:01.429940 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2817849-b374-4498-9103-f5ffb9806074 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:01.430212 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2817849-b374-4498-9103-f5ffb9806074 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:01.430482 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2817849-b374-4498-9103-f5ffb9806074 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:01.437928 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 258/512] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:31:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:31:01.438805 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5052687d-f4d8-4ab2-b81f-2cd38c740350 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:01.445909 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2817849-b374-4498-9103-f5ffb9806074 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:01.447386 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2817849-b374-4498-9103-f5ffb9806074 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:01.464433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-b2817849-b374-4498-9103-f5ffb9806074 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:01.464433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:01.464433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:01.464433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:01.464433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:01.464433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:01.464433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:01.464433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:01.464433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:01.464433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:01.464433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:01.464433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:01.464433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:01.464433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:01.464433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:01.464433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:01.465502 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:01.465502 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:01.465502 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:01.465502 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:01.465502 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:01.465502 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:01.465502 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:01.465502 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:01.465502 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:01.465502 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:01.465502 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:01.465502 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:01.465502 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:01.465502 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:01.465502 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:01.465502 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:01.465502 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:01.466693 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:01.466693 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:01.466693 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:01.466693 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 255/513] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:01.466693 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2817849-b374-4498-9103-f5ffb9806074 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:01.528715 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:01.540968 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5052687d-f4d8-4ab2-b81f-2cd38c740350 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:01.563023 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2817849-b374-4498-9103-f5ffb9806074 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:01.565781 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:01.575307 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859da834-72bb-46e0-ad45-f843e400f180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:01.575905 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859da834-72bb-46e0-ad45-f843e400f180 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:01.576219 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859da834-72bb-46e0-ad45-f843e400f180 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:01.576487 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859da834-72bb-46e0-ad45-f843e400f180 None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:01.577126 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859da834-72bb-46e0-ad45-f843e400f180 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:01.577428 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859da834-72bb-46e0-ad45-f843e400f180 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:01.578812 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859da834-72bb-46e0-ad45-f843e400f180 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:01.590942 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 259/514] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:01.591719 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859da834-72bb-46e0-ad45-f843e400f180 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:01.596426 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41bf577-0b05-4f30-86b9-bcbedcd1124c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:01.597049 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41bf577-0b05-4f30-86b9-bcbedcd1124c None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:01.597325 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41bf577-0b05-4f30-86b9-bcbedcd1124c None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:01.597594 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41bf577-0b05-4f30-86b9-bcbedcd1124c None admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:01.598322 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41bf577-0b05-4f30-86b9-bcbedcd1124c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:01.598627 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41bf577-0b05-4f30-86b9-bcbedcd1124c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:01.600076 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41bf577-0b05-4f30-86b9-bcbedcd1124c None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:01.616909 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 256/515] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Aug 17 23:31:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:01.617693 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41bf577-0b05-4f30-86b9-bcbedcd1124c None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:01.691609 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859da834-72bb-46e0-ad45-f843e400f180 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:01.716715 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41bf577-0b05-4f30-86b9-bcbedcd1124c None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:01.725250 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff6d8d4-6124-4213-9239-aeeec5f10961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:01.725928 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6d8d4-6124-4213-9239-aeeec5f10961 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:01.726208 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6d8d4-6124-4213-9239-aeeec5f10961 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:01.726478 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6d8d4-6124-4213-9239-aeeec5f10961 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:01.737148 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff6d8d4-6124-4213-9239-aeeec5f10961 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:01.738598 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff6d8d4-6124-4213-9239-aeeec5f10961 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:01.739929 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:01.751341 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9be71-bc0f-47fe-837a-a29ffca0b434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:01.751984 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9be71-bc0f-47fe-837a-a29ffca0b434 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:01.752267 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9be71-bc0f-47fe-837a-a29ffca0b434 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:01.752542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9be71-bc0f-47fe-837a-a29ffca0b434 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:01.752629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-8ff6d8d4-6124-4213-9239-aeeec5f10961 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:01.752629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:01.752629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:01.752629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:01.752629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:01.752629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:01.752629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:01.752629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:01.752629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:01.752629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:01.752629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:01.752629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:01.752629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:01.752629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:01.752629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:01.752629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:01.753923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:01.753923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:01.753923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:01.753923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:01.753923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:01.753923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:01.753923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:01.753923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:01.753923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:01.753923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:01.753923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:01.753923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:01.753923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:01.753923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:01.753923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:01.753923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:01.753923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:01.754775 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:01.754775 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:01.754775 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:01.754775 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 260/516] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:01.754775 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff6d8d4-6124-4213-9239-aeeec5f10961 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:01.765286 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb9be71-bc0f-47fe-837a-a29ffca0b434 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:01.766741 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb9be71-bc0f-47fe-837a-a29ffca0b434 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:01.781886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-dbb9be71-bc0f-47fe-837a-a29ffca0b434 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:01.781886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:01.781886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:01.781886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:01.781886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:01.781886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:01.781886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:01.781886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:01.781886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:01.781886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:01.781886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:01.781886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:01.781886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:01.781886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:01.781886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:01.781886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:01.782825 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:01.782825 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:01.782825 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:01.782825 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:01.782825 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:01.782825 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:01.782825 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:01.782825 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:01.782825 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:01.782825 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:01.782825 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:01.782825 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:01.782825 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:01.782825 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:01.782825 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:01.782825 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:01.782825 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:01.783877 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:01.783877 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:01.783877 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:01.783877 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 257/517] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:01.783877 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9be71-bc0f-47fe-837a-a29ffca0b434 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:01.798346 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:01.856511 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff6d8d4-6124-4213-9239-aeeec5f10961 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:01.884230 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9be71-bc0f-47fe-837a-a29ffca0b434 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:01.889245 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e9fb4c-0f66-4bde-a9a2-dbeeb191c169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:01.889848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9fb4c-0f66-4bde-a9a2-dbeeb191c169 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:01.890128 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9fb4c-0f66-4bde-a9a2-dbeeb191c169 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:01.890400 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9fb4c-0f66-4bde-a9a2-dbeeb191c169 None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:01.891034 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e9fb4c-0f66-4bde-a9a2-dbeeb191c169 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:01.891340 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e9fb4c-0f66-4bde-a9a2-dbeeb191c169 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:01.892739 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e9fb4c-0f66-4bde-a9a2-dbeeb191c169 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:01.904385 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 261/518] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:01.905167 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e9fb4c-0f66-4bde-a9a2-dbeeb191c169 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:01.920461 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1079b25-5403-4d61-9382-db14478dd3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:01.921057 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1079b25-5403-4d61-9382-db14478dd3aa None admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:01.921336 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1079b25-5403-4d61-9382-db14478dd3aa None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:01.921603 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1079b25-5403-4d61-9382-db14478dd3aa None admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:01.922237 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1079b25-5403-4d61-9382-db14478dd3aa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:01.923677 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1079b25-5403-4d61-9382-db14478dd3aa None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:01.945388 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 258/519] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Aug 17 23:31:01 2022] POST /identity/v3/users => generated 247 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:31:01.946189 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1079b25-5403-4d61-9382-db14478dd3aa None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:01.982088 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:02.015237 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:02.019758 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e9fb4c-0f66-4bde-a9a2-dbeeb191c169 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:02.059551 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad525fc-d3b6-4be3-8979-65191e8c4239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:02.060203 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad525fc-d3b6-4be3-8979-65191e8c4239 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:02.060485 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad525fc-d3b6-4be3-8979-65191e8c4239 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:02.060755 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad525fc-d3b6-4be3-8979-65191e8c4239 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:02.068924 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1079b25-5403-4d61-9382-db14478dd3aa None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:02.107939 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63205e64-d4b6-4bf1-9a7e-45e7ac114abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:02.108578 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63205e64-d4b6-4bf1-9a7e-45e7ac114abb None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:02.108853 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63205e64-d4b6-4bf1-9a7e-45e7ac114abb None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:02.109125 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63205e64-d4b6-4bf1-9a7e-45e7ac114abb None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:02.109765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63205e64-d4b6-4bf1-9a7e-45e7ac114abb None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:02.110064 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63205e64-d4b6-4bf1-9a7e-45e7ac114abb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:02.111502 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63205e64-d4b6-4bf1-9a7e-45e7ac114abb None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:02.125996 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 259/520] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:02.130180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63205e64-d4b6-4bf1-9a7e-45e7ac114abb None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:02.144976 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad525fc-d3b6-4be3-8979-65191e8c4239 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:02.146524 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad525fc-d3b6-4be3-8979-65191e8c4239 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:02.234535 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:02.243598 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63205e64-d4b6-4bf1-9a7e-45e7ac114abb None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:02.251760 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:02.257241 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 262/521] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:31:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:31:02.258314 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad525fc-d3b6-4be3-8979-65191e8c4239 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:02.282505 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b3006-3e7e-48ed-b2ce-6511c7904866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:02.283112 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b3006-3e7e-48ed-b2ce-6511c7904866 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:02.283389 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b3006-3e7e-48ed-b2ce-6511c7904866 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:02.283661 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b3006-3e7e-48ed-b2ce-6511c7904866 None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:02.284338 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3b3006-3e7e-48ed-b2ce-6511c7904866 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:02.284638 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3b3006-3e7e-48ed-b2ce-6511c7904866 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:02.286013 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3b3006-3e7e-48ed-b2ce-6511c7904866 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:02.302508 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 260/522] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:02.302508 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b3006-3e7e-48ed-b2ce-6511c7904866 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:02.370024 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad525fc-d3b6-4be3-8979-65191e8c4239 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:02.399272 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b3006-3e7e-48ed-b2ce-6511c7904866 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:02.404560 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449f705-b605-4358-a900-c11180e4492d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:02.405187 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449f705-b605-4358-a900-c11180e4492d None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:02.405464 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449f705-b605-4358-a900-c11180e4492d None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:02.405733 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449f705-b605-4358-a900-c11180e4492d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:02.412934 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a449f705-b605-4358-a900-c11180e4492d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:02.414397 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a449f705-b605-4358-a900-c11180e4492d None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:02.426228 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-a449f705-b605-4358-a900-c11180e4492d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 17 23:31:02.426228 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:02.426228 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:02.426228 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:02.426228 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:02.426228 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:02.426228 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:02.426228 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:02.426228 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:02.426228 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:02.426228 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:02.426228 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:02.426228 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 23:31:02.426228 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 23:31:02.426228 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 23:31:02.426228 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 23:31:02.427157 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:02.427157 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:02.427157 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 23:31:02.427157 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:02.427157 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 23:31:02.427157 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:02.427157 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:02.427157 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:02.427157 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:02.427157 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:02.427157 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:02.427157 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:02.427157 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:02.427157 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:02.427157 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:02.427157 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:02.427157 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:02.427157 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:02.428104 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:02.428104 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:02.428104 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 23:31:02.428104 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 23:31:02.428104 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 23:31:02.428104 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 23:31:02.428104 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 23:31:02.428104 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 23:31:02.428104 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 17 23:31:02.428104 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:02.428104 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 263/523] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 17 23:31:02 2022] GET /identity/v3/users/neutron => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:02.428674 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449f705-b605-4358-a900-c11180e4492d None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:02.440413 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719fb61d-27a3-456f-9ad8-dd0c6d33d61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:02.441244 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719fb61d-27a3-456f-9ad8-dd0c6d33d61e None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:02.441658 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719fb61d-27a3-456f-9ad8-dd0c6d33d61e None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:02.442212 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719fb61d-27a3-456f-9ad8-dd0c6d33d61e None admin] PATH_INFO: `/v3/projects/service` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:02.449809 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719fb61d-27a3-456f-9ad8-dd0c6d33d61e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:02.451428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719fb61d-27a3-456f-9ad8-dd0c6d33d61e None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:02.464521 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-719fb61d-27a3-456f-9ad8-dd0c6d33d61e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:02.464521 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:02.464521 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:02.464521 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:02.464521 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:02.464521 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:02.464521 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:02.464521 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:02.464521 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:02.464521 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:02.464521 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:02.464521 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:02.464521 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 23:31:02.464521 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 23:31:02.464521 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 23:31:02.465584 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 23:31:02.465584 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:02.465584 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:02.465584 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:02.465584 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:02.465584 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:02.465584 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:02.465584 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:02.465584 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:02.465584 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:02.465584 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:02.465584 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:02.465584 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:02.465584 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:02.465584 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:02.465584 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:02.465584 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:02.466568 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 23:31:02.466568 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 23:31:02.466568 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 23:31:02.466568 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 23:31:02.466568 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 23:31:02.466568 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 23:31:02.466568 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:02.466568 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:02.466568 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 261/524] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:31:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:02.469715 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719fb61d-27a3-456f-9ad8-dd0c6d33d61e None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:02.488130 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:02.496733 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:02.541600 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449f705-b605-4358-a900-c11180e4492d None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:02.577765 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91fba2-e948-4093-9c18-fbb0d8a67f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:02.578622 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91fba2-e948-4093-9c18-fbb0d8a67f56 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:02.578907 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91fba2-e948-4093-9c18-fbb0d8a67f56 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:02.579180 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91fba2-e948-4093-9c18-fbb0d8a67f56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:02.581068 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719fb61d-27a3-456f-9ad8-dd0c6d33d61e None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:02.590665 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91fba2-e948-4093-9c18-fbb0d8a67f56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:02.592161 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91fba2-e948-4093-9c18-fbb0d8a67f56 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:02.607107 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-bc91fba2-e948-4093-9c18-fbb0d8a67f56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:02.607107 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:02.607107 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:02.607107 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:02.607107 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:02.607107 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:02.607107 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:02.607107 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:02.607107 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:02.607107 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:02.607107 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:02.607107 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:02.607107 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:02.607107 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:02.607107 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:02.607107 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:02.608160 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:02.608160 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:02.608160 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:02.608160 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:02.608160 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:02.608160 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:02.608160 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:02.608160 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:02.608160 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:02.608160 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:02.608160 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:02.608160 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:02.608160 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:02.608160 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:02.608160 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:02.608160 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:02.608160 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:02.609010 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:02.609010 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:02.609010 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:02.609010 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 264/525] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:02.609010 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91fba2-e948-4093-9c18-fbb0d8a67f56 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:02.626655 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef24685c-b545-4162-9188-3f7b7b22c1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:02.627419 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef24685c-b545-4162-9188-3f7b7b22c1e2 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:02.627799 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef24685c-b545-4162-9188-3f7b7b22c1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:02.628119 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef24685c-b545-4162-9188-3f7b7b22c1e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:02.634914 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef24685c-b545-4162-9188-3f7b7b22c1e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:02.636462 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef24685c-b545-4162-9188-3f7b7b22c1e2 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:02.647533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-ef24685c-b545-4162-9188-3f7b7b22c1e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:02.647533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:02.647533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:02.647533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:02.647533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:02.647533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:02.647533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:02.647533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:02.647533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:02.647533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:02.647533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:02.647533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:02.647533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 23:31:02.647533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 23:31:02.647533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 23:31:02.648608 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 23:31:02.648608 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:02.648608 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:02.648608 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:02.648608 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:02.648608 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:02.648608 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:02.648608 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:02.648608 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:02.648608 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:02.648608 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:02.648608 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:02.648608 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:02.648608 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:02.648608 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:02.648608 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:02.648608 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:02.649583 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 23:31:02.649583 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 23:31:02.649583 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 23:31:02.649583 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 23:31:02.649583 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 23:31:02.649583 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 23:31:02.649583 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:02.649583 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:02.649583 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 262/526] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:31:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:02.650057 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef24685c-b545-4162-9188-3f7b7b22c1e2 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:02.721009 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91fba2-e948-4093-9c18-fbb0d8a67f56 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:02.731315 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:02.734295 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:02.760026 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5764bd0-39d3-4364-adf6-8752b14b0182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:02.760697 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5764bd0-39d3-4364-adf6-8752b14b0182 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:02.761531 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5764bd0-39d3-4364-adf6-8752b14b0182 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:02.761802 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5764bd0-39d3-4364-adf6-8752b14b0182 None admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:02.762517 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5764bd0-39d3-4364-adf6-8752b14b0182 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:02.762819 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5764bd0-39d3-4364-adf6-8752b14b0182 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:02.764257 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef24685c-b545-4162-9188-3f7b7b22c1e2 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:02.764359 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5764bd0-39d3-4364-adf6-8752b14b0182 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:02.782658 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 265/527] 10.222.0.26 () {56 vars in 1139 bytes} [Wed Aug 17 23:31:02 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:02.783528 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5764bd0-39d3-4364-adf6-8752b14b0182 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:02.804974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7365cebe-a856-4012-876d-b72ba6a62b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:02.806040 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365cebe-a856-4012-876d-b72ba6a62b7a None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:02.806410 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365cebe-a856-4012-876d-b72ba6a62b7a None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:02.806752 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365cebe-a856-4012-876d-b72ba6a62b7a None admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:02.807640 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7365cebe-a856-4012-876d-b72ba6a62b7a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:02.808132 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7365cebe-a856-4012-876d-b72ba6a62b7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:02.809640 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7365cebe-a856-4012-876d-b72ba6a62b7a None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:02.827663 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 263/528] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 17 23:31:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:02.828678 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7365cebe-a856-4012-876d-b72ba6a62b7a None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:02.896015 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5764bd0-39d3-4364-adf6-8752b14b0182 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:02.928990 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78241a6f-9e7f-4fea-8732-1a416bf08e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:02.929592 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78241a6f-9e7f-4fea-8732-1a416bf08e48 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:02.929865 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78241a6f-9e7f-4fea-8732-1a416bf08e48 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:02.930138 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78241a6f-9e7f-4fea-8732-1a416bf08e48 None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:02.930772 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78241a6f-9e7f-4fea-8732-1a416bf08e48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:02.931068 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78241a6f-9e7f-4fea-8732-1a416bf08e48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:02.932493 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78241a6f-9e7f-4fea-8732-1a416bf08e48 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:02.939125 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7365cebe-a856-4012-876d-b72ba6a62b7a None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:02.948483 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 266/529] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:02.948483 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78241a6f-9e7f-4fea-8732-1a416bf08e48 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:02.972349 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:02.973641 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:02.979836 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6dd84b-6297-4b54-8609-f151be2f4ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:02.980602 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6dd84b-6297-4b54-8609-f151be2f4ccb None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:02.980988 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6dd84b-6297-4b54-8609-f151be2f4ccb None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:02.981355 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6dd84b-6297-4b54-8609-f151be2f4ccb None admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:02.982143 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6dd84b-6297-4b54-8609-f151be2f4ccb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:02.982509 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6dd84b-6297-4b54-8609-f151be2f4ccb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:02.984069 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6dd84b-6297-4b54-8609-f151be2f4ccb None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:02.997448 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 264/530] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 17 23:31:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:02.998074 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6dd84b-6297-4b54-8609-f151be2f4ccb None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:03.057522 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78241a6f-9e7f-4fea-8732-1a416bf08e48 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:03.095641 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e4c12-792a-4184-ac9a-04179c23694e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:03.096491 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e4c12-792a-4184-ac9a-04179c23694e None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:03.096849 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e4c12-792a-4184-ac9a-04179c23694e None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:03.097176 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e4c12-792a-4184-ac9a-04179c23694e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:03.101794 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6dd84b-6297-4b54-8609-f151be2f4ccb None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:03.134658 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3344b7-411f-47f3-9918-b7a55eaa8200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:03.135331 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3344b7-411f-47f3-9918-b7a55eaa8200 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:03.135678 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3344b7-411f-47f3-9918-b7a55eaa8200 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:03.136114 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3344b7-411f-47f3-9918-b7a55eaa8200 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:03.136899 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3344b7-411f-47f3-9918-b7a55eaa8200 None admin] RBAC: Adding query filter params (role.id=b64f80e6f3ea4ad49b94acdde42ccd87, scope.project.id=4de86915ef4340248ba8a8f51d56594e, user.id=9c08c497a14c426f9cfe0dd8c8100a21) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:03.137315 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3344b7-411f-47f3-9918-b7a55eaa8200 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:03.138805 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3344b7-411f-47f3-9918-b7a55eaa8200 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:03.144246 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 265/531] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:31:02 2022] GET /identity/v3/role_assignments?user.id=9c08c497a14c426f9cfe0dd8c8100a21&scope.project.id=4de86915ef4340248ba8a8f51d56594e&role.id=b64f80e6f3ea4ad49b94acdde42ccd87 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:03.145101 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3344b7-411f-47f3-9918-b7a55eaa8200 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:03.190001 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e4c12-792a-4184-ac9a-04179c23694e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:03.191741 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e4c12-792a-4184-ac9a-04179c23694e None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:03.223615 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:03.233660 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:03.300161 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3344b7-411f-47f3-9918-b7a55eaa8200 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:03.326220 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 267/532] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:31:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:31:03.327406 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e4c12-792a-4184-ac9a-04179c23694e None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:03.347804 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae112b-320d-4606-8954-3f9b386ef6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:03.348472 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae112b-320d-4606-8954-3f9b386ef6b2 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:03.348749 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae112b-320d-4606-8954-3f9b386ef6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:03.349070 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae112b-320d-4606-8954-3f9b386ef6b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:03.358527 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae112b-320d-4606-8954-3f9b386ef6b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:03.360538 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae112b-320d-4606-8954-3f9b386ef6b2 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:03.375605 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-72ae112b-320d-4606-8954-3f9b386ef6b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:03.375605 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:03.375605 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:03.375605 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:03.375605 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:03.375605 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:03.375605 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:03.375605 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:03.375605 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:03.375605 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:03.375605 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:03.375605 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:03.375605 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 23:31:03.375605 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 23:31:03.375605 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 23:31:03.376534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 23:31:03.376534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:03.376534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:03.376534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:03.376534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:03.376534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:03.376534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:03.376534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:03.376534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:03.376534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:03.376534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:03.376534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:03.376534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:03.376534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:03.376534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:03.376534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:03.376534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:03.377377 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 23:31:03.377377 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 23:31:03.377377 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 23:31:03.377377 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 23:31:03.377377 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 23:31:03.377377 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 23:31:03.377377 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:03.377377 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:03.380042 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 266/533] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:31:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:03.381092 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae112b-320d-4606-8954-3f9b386ef6b2 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:03.464810 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e4c12-792a-4184-ac9a-04179c23694e None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:03.493137 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:03.501565 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae112b-320d-4606-8954-3f9b386ef6b2 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:03.503707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7c02c-eece-4448-8501-3b7307a0e2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:03.504524 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7c02c-eece-4448-8501-3b7307a0e2b2 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:03.504945 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7c02c-eece-4448-8501-3b7307a0e2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:03.505352 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7c02c-eece-4448-8501-3b7307a0e2b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:03.506206 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec7c02c-eece-4448-8501-3b7307a0e2b2 None admin] RBAC: Adding query filter params (role.id=b64f80e6f3ea4ad49b94acdde42ccd87, scope.project.id=4de86915ef4340248ba8a8f51d56594e, user.id=62906d50b6e247c59b69809d3959d4de) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:03.506650 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec7c02c-eece-4448-8501-3b7307a0e2b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:03.508253 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec7c02c-eece-4448-8501-3b7307a0e2b2 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:03.520264 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 268/534] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:31:03 2022] GET /identity/v3/role_assignments?user.id=62906d50b6e247c59b69809d3959d4de&scope.project.id=4de86915ef4340248ba8a8f51d56594e&role.id=b64f80e6f3ea4ad49b94acdde42ccd87 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:03.520648 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:03.521989 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7c02c-eece-4448-8501-3b7307a0e2b2 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:03.544020 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587bdaf4-8764-4062-9678-f4f55e45f224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:03.544020 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bdaf4-8764-4062-9678-f4f55e45f224 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:03.544020 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bdaf4-8764-4062-9678-f4f55e45f224 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:03.544020 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bdaf4-8764-4062-9678-f4f55e45f224 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:03.563993 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587bdaf4-8764-4062-9678-f4f55e45f224 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:03.563993 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587bdaf4-8764-4062-9678-f4f55e45f224 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:03.574589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-587bdaf4-8764-4062-9678-f4f55e45f224 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:03.574589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:03.574589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:03.574589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:03.574589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:03.574589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:03.574589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:03.574589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:03.574589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:03.574589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:03.574589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:03.574589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:03.574589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:03.574589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:03.574589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:03.574589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:03.575713 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:03.575713 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:03.575713 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:03.575713 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:03.575713 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:03.575713 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:03.575713 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:03.575713 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:03.575713 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:03.575713 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:03.575713 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:03.575713 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:03.575713 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:03.575713 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:03.575713 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:03.575713 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:03.575713 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:03.576829 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:03.576829 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:03.576829 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:03.578136 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 267/535] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:03.588190 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587bdaf4-8764-4062-9678-f4f55e45f224 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:03.660527 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7c02c-eece-4448-8501-3b7307a0e2b2 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:03.697870 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150610eb-191c-4780-a3f1-9c45e67ca5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:03.699297 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150610eb-191c-4780-a3f1-9c45e67ca5f9 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:03.699714 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150610eb-191c-4780-a3f1-9c45e67ca5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:03.700153 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150610eb-191c-4780-a3f1-9c45e67ca5f9 None admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:03.700578 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587bdaf4-8764-4062-9678-f4f55e45f224 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:03.701743 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150610eb-191c-4780-a3f1-9c45e67ca5f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:03.702337 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150610eb-191c-4780-a3f1-9c45e67ca5f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:03.704917 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150610eb-191c-4780-a3f1-9c45e67ca5f9 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:03.746231 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 269/536] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 17 23:31:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:03.761074 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150610eb-191c-4780-a3f1-9c45e67ca5f9 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:03.761817 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea41772-4cb2-44f8-847e-7ca9b06ec704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:03.762890 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea41772-4cb2-44f8-847e-7ca9b06ec704 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:03.763333 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea41772-4cb2-44f8-847e-7ca9b06ec704 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:03.763737 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea41772-4cb2-44f8-847e-7ca9b06ec704 None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:03.764830 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea41772-4cb2-44f8-847e-7ca9b06ec704 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:03.765260 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea41772-4cb2-44f8-847e-7ca9b06ec704 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:03.766915 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea41772-4cb2-44f8-847e-7ca9b06ec704 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:03.783002 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 268/537] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:03.789704 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea41772-4cb2-44f8-847e-7ca9b06ec704 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:03.825024 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:03.852481 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:03.920544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea41772-4cb2-44f8-847e-7ca9b06ec704 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:03.925721 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150610eb-191c-4780-a3f1-9c45e67ca5f9 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:03.957743 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92f2db6-fd8a-4735-995b-e843e1929c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:03.958513 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f2db6-fd8a-4735-995b-e843e1929c87 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:03.958931 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f2db6-fd8a-4735-995b-e843e1929c87 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:03.959474 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f2db6-fd8a-4735-995b-e843e1929c87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:03.968773 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92f2db6-fd8a-4735-995b-e843e1929c87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:03.970217 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92f2db6-fd8a-4735-995b-e843e1929c87 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:03.975177 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a66ff8c-50f9-42e0-99e4-1c31f3d39428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:03.975779 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66ff8c-50f9-42e0-99e4-1c31f3d39428 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:03.976086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66ff8c-50f9-42e0-99e4-1c31f3d39428 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:03.976360 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66ff8c-50f9-42e0-99e4-1c31f3d39428 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:03.977077 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a66ff8c-50f9-42e0-99e4-1c31f3d39428 None admin] RBAC: Adding query filter params (role.id=b64f80e6f3ea4ad49b94acdde42ccd87, scope.project.id=4de86915ef4340248ba8a8f51d56594e, user.id=ec1dcbe43b0749c3bcc4af9100d771d1) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:03.977391 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a66ff8c-50f9-42e0-99e4-1c31f3d39428 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:03.978808 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a66ff8c-50f9-42e0-99e4-1c31f3d39428 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:03.985264 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 270/538] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:31:03 2022] GET /identity/v3/role_assignments?user.id=ec1dcbe43b0749c3bcc4af9100d771d1&scope.project.id=4de86915ef4340248ba8a8f51d56594e&role.id=b64f80e6f3ea4ad49b94acdde42ccd87 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:03.986421 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58ddb5-abb9-45e7-9412-60f05fe65958 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:03.986703 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58ddb5-abb9-45e7-9412-60f05fe65958 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:03.986972 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58ddb5-abb9-45e7-9412-60f05fe65958 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:03.989744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-b92f2db6-fd8a-4735-995b-e843e1929c87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:03.989744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:03.989744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:03.989744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:03.989744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:03.989744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:03.989744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:03.989744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:03.989744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:03.989744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:03.989744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:03.989744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:03.989744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 23:31:03.989744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 23:31:03.989744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 23:31:03.990557 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 23:31:03.990557 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:03.990557 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:03.990557 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:03.990557 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:03.990557 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:03.990557 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:03.990557 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:03.990557 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:03.990557 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:03.990557 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:03.990557 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:03.990557 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:03.990557 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:03.990557 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:03.990557 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:03.990557 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:03.991422 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 23:31:03.991422 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 23:31:03.991422 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 23:31:03.991422 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 23:31:03.991422 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 23:31:03.991422 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 23:31:03.991422 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:03.991422 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:03.993978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 269/539] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:31:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:03.996581 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 271/540] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:03 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:03.999255 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92f2db6-fd8a-4735-995b-e843e1929c87 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:04.006024 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297192db-142f-4e43-813a-f9f6bc170af7 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:04.006668 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297192db-142f-4e43-813a-f9f6bc170af7 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:04.007161 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297192db-142f-4e43-813a-f9f6bc170af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:04.034736 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-297192db-142f-4e43-813a-f9f6bc170af7 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:04.036387 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-297192db-142f-4e43-813a-f9f6bc170af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:04.122914 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 272/541] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:31:04.177245 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:04.191257 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92f2db6-fd8a-4735-995b-e843e1929c87 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:04.228557 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64ef46d-ae23-40a4-b628-616594c09f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:04.229511 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ef46d-ae23-40a4-b628-616594c09f84 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:04.230073 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ef46d-ae23-40a4-b628-616594c09f84 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:04.230556 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ef46d-ae23-40a4-b628-616594c09f84 None admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:04.231556 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64ef46d-ae23-40a4-b628-616594c09f84 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:04.232080 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64ef46d-ae23-40a4-b628-616594c09f84 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:04.233718 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64ef46d-ae23-40a4-b628-616594c09f84 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:04.246968 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 270/542] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 17 23:31:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:04.252397 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297192db-142f-4e43-813a-f9f6bc170af7 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:04.307820 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:04.363846 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297192db-142f-4e43-813a-f9f6bc170af7 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:04.399851 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4953ee91-48e9-43bc-b6a8-b3f201a58738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:04.400820 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953ee91-48e9-43bc-b6a8-b3f201a58738 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:04.401243 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953ee91-48e9-43bc-b6a8-b3f201a58738 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:04.401702 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953ee91-48e9-43bc-b6a8-b3f201a58738 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:04.402581 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4953ee91-48e9-43bc-b6a8-b3f201a58738 None admin] RBAC: Adding query filter params (role.id=b64f80e6f3ea4ad49b94acdde42ccd87, scope.project.id=4de86915ef4340248ba8a8f51d56594e, user.id=16e2758f1cd144a98b98d9be0fa46ae2) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:04.403335 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4953ee91-48e9-43bc-b6a8-b3f201a58738 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:04.405059 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4953ee91-48e9-43bc-b6a8-b3f201a58738 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:04.411149 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 273/543] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:31:04 2022] GET /identity/v3/role_assignments?user.id=16e2758f1cd144a98b98d9be0fa46ae2&scope.project.id=4de86915ef4340248ba8a8f51d56594e&role.id=b64f80e6f3ea4ad49b94acdde42ccd87 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:04.493111 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64ef46d-ae23-40a4-b628-616594c09f84 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:04.601369 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64ef46d-ae23-40a4-b628-616594c09f84 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:04.640332 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cceafd4-3e83-416d-b033-fefb27567d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:04.641214 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cceafd4-3e83-416d-b033-fefb27567d31 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:04.641640 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cceafd4-3e83-416d-b033-fefb27567d31 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:04.642087 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cceafd4-3e83-416d-b033-fefb27567d31 None admin] PATH_INFO: `/v3/roles/service` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:04.653889 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cceafd4-3e83-416d-b033-fefb27567d31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:31:04.658951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-0cceafd4-3e83-416d-b033-fefb27567d31 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:31:04.658951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:04.658951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:04.658951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:04.658951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:04.658951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:04.658951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:04.658951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:04.658951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:04.658951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:04.658951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:04.658951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:04.658951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:31:04.658951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:31:04.658951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:31:04.658951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:31:04.660110 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:31:04.660110 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:31:04.660110 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:31:04.660110 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:31:04.660110 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:04.660110 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:04.660110 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:04.660110 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:04.660110 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:04.660110 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:04.660110 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:04.660110 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:04.660110 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:04.660110 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:04.660110 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:04.660110 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:04.661088 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:04.661088 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:04.661088 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:04.661088 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:04.661088 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:31:04.661088 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:31:04.661088 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:31:04.661088 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:31:04.661088 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:31:04.661088 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:31:04.661088 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:31:04.661088 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:04.663345 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 271/544] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 17 23:31:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:04.669413 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4953ee91-48e9-43bc-b6a8-b3f201a58738 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:04.780169 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4953ee91-48e9-43bc-b6a8-b3f201a58738 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:04.796515 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:04.829470 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1dadee-d320-4e30-aae9-cc67df78ac0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:04.830091 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1dadee-d320-4e30-aae9-cc67df78ac0c None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:04.830373 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1dadee-d320-4e30-aae9-cc67df78ac0c None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:04.830646 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1dadee-d320-4e30-aae9-cc67df78ac0c None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:04.831316 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1dadee-d320-4e30-aae9-cc67df78ac0c None admin] RBAC: Adding query filter params (name=service) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:04.831623 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1dadee-d320-4e30-aae9-cc67df78ac0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:04.833097 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1dadee-d320-4e30-aae9-cc67df78ac0c None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:04.847645 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 274/545] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 17 23:31:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:04.853209 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cceafd4-3e83-416d-b033-fefb27567d31 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:04.868384 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:05.002794 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cceafd4-3e83-416d-b033-fefb27567d31 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:05.057194 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c9473-afd1-43db-988d-086954919d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:05.057875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c9473-afd1-43db-988d-086954919d5b None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:05.058156 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c9473-afd1-43db-988d-086954919d5b None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:05.058460 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c9473-afd1-43db-988d-086954919d5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:05.068475 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291c9473-afd1-43db-988d-086954919d5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:05.069908 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291c9473-afd1-43db-988d-086954919d5b None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:05.085071 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-291c9473-afd1-43db-988d-086954919d5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:05.085071 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:05.085071 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:05.085071 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:05.085071 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:05.085071 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:05.085071 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:05.085071 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:05.085071 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:05.085071 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:05.085071 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:05.085071 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:05.085071 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:05.085071 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:05.085071 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:05.085071 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:05.086031 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:05.086031 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:05.086031 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:05.086031 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:05.086031 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:05.086031 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:05.086031 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:05.086031 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:05.086031 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:05.086031 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:05.086031 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:05.086031 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:05.086031 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:05.086031 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:05.086031 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:05.086031 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:05.086031 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:05.086886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:05.086886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:05.086886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:05.086886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 272/546] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:05.093648 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1dadee-d320-4e30-aae9-cc67df78ac0c None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:05.220141 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1dadee-d320-4e30-aae9-cc67df78ac0c None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:05.256570 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b028f3e4-c107-4091-8fb5-91d3aa6f62fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:05.257100 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028f3e4-c107-4091-8fb5-91d3aa6f62fb None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:05.257379 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028f3e4-c107-4091-8fb5-91d3aa6f62fb None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:05.257648 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028f3e4-c107-4091-8fb5-91d3aa6f62fb None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:05.258290 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b028f3e4-c107-4091-8fb5-91d3aa6f62fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:05.258594 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b028f3e4-c107-4091-8fb5-91d3aa6f62fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:05.259955 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b028f3e4-c107-4091-8fb5-91d3aa6f62fb None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:05.272823 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 275/547] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:05.278918 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c9473-afd1-43db-988d-086954919d5b None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:05.292800 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5897f-1e74-4bb4-b145-14cb40f0b8a6 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:05.293079 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5897f-1e74-4bb4-b145-14cb40f0b8a6 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:05.293359 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5897f-1e74-4bb4-b145-14cb40f0b8a6 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:05.294968 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 276/548] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:05.300289 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a8857-797e-4cbe-97c8-4193c5ee3dea None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:05.300570 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a8857-797e-4cbe-97c8-4193c5ee3dea None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:05.300841 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a8857-797e-4cbe-97c8-4193c5ee3dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:05.333915 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-b84a8857-797e-4cbe-97c8-4193c5ee3dea None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:05.335129 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-b84a8857-797e-4cbe-97c8-4193c5ee3dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:05.369420 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:05.388071 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:05.444194 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 277/549] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 23:31:05.454613 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c9473-afd1-43db-988d-086954919d5b None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:05.490098 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b2ade-1026-4095-a2d9-5353fd4ca957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:05.490702 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b2ade-1026-4095-a2d9-5353fd4ca957 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:05.490978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b2ade-1026-4095-a2d9-5353fd4ca957 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:05.491247 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b2ade-1026-4095-a2d9-5353fd4ca957 None admin] PATH_INFO: `/v3/users/swift` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:05.499512 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b2ade-1026-4095-a2d9-5353fd4ca957 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:05.500973 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b2ade-1026-4095-a2d9-5353fd4ca957 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:05.512344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-179b2ade-1026-4095-a2d9-5353fd4ca957 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 17 23:31:05.512344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:05.512344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:05.512344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:05.512344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:05.512344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:05.512344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:05.512344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:05.512344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:05.512344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:05.512344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:05.512344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:05.512344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 23:31:05.512344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 23:31:05.512344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 23:31:05.512344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 23:31:05.513759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:05.513759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:05.513759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 23:31:05.513759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:05.513759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 23:31:05.513759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:05.513759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:05.513759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:05.513759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:05.513759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:05.513759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:05.513759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:05.513759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:05.513759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:05.513759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:05.513759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:05.513759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:05.513759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:05.514863 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:05.514863 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:05.514863 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 23:31:05.514863 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 23:31:05.514863 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 23:31:05.514863 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 23:31:05.514863 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 23:31:05.514863 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 23:31:05.514863 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 17 23:31:05.514863 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:05.514863 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 273/550] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 17 23:31:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:05.521280 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a8857-797e-4cbe-97c8-4193c5ee3dea None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:05.648594 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a8857-797e-4cbe-97c8-4193c5ee3dea None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:05.691911 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efad44f-fb22-4424-9774-7e7bba679328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:05.692545 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efad44f-fb22-4424-9774-7e7bba679328 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:05.692819 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efad44f-fb22-4424-9774-7e7bba679328 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:05.693084 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efad44f-fb22-4424-9774-7e7bba679328 None admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:05.693804 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efad44f-fb22-4424-9774-7e7bba679328 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:05.694105 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efad44f-fb22-4424-9774-7e7bba679328 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:05.695526 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efad44f-fb22-4424-9774-7e7bba679328 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:05.715414 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 278/551] 10.222.0.26 () {56 vars in 1135 bytes} [Wed Aug 17 23:31:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:05.725027 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b2ade-1026-4095-a2d9-5353fd4ca957 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:05.782506 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efad44f-fb22-4424-9774-7e7bba679328 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:05.808555 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:05.897084 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efad44f-fb22-4424-9774-7e7bba679328 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:05.921686 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b2ade-1026-4095-a2d9-5353fd4ca957 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:05.935418 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be7d2f0-9f03-4699-a364-407bf1f9247e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:05.935418 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7d2f0-9f03-4699-a364-407bf1f9247e None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:05.935418 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7d2f0-9f03-4699-a364-407bf1f9247e None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:05.935418 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7d2f0-9f03-4699-a364-407bf1f9247e None admin] PATH_INFO: `/v3/roles/service` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:05.948392 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be7d2f0-9f03-4699-a364-407bf1f9247e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:31:05.952752 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-2be7d2f0-9f03-4699-a364-407bf1f9247e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:31:05.952752 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:05.952752 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:05.952752 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:05.952752 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:05.952752 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:05.952752 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:05.952752 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:05.952752 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:05.952752 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:05.952752 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:05.952752 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:05.952752 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:31:05.952752 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:31:05.952752 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:31:05.952752 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:31:05.953882 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:31:05.953882 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:31:05.953882 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:31:05.953882 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:31:05.953882 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:05.953882 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:05.953882 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:05.953882 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:05.953882 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:05.953882 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:05.953882 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:05.953882 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:05.953882 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:05.953882 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:05.953882 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:05.953882 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:05.955113 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:05.955113 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:05.955113 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:05.955113 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:05.955113 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:31:05.955113 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:31:05.955113 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:31:05.955113 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:31:05.955113 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:31:05.955113 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:31:05.955113 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:31:05.955113 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:05.956910 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 279/552] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 17 23:31:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:31:05.958892 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:05.962799 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be7d2f0-9f03-4699-a364-407bf1f9247e None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:05.993318 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee5ee2-0ac4-4af7-a4f0-2207636fce89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:05.993937 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee5ee2-0ac4-4af7-a4f0-2207636fce89 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:05.994210 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee5ee2-0ac4-4af7-a4f0-2207636fce89 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:05.994487 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee5ee2-0ac4-4af7-a4f0-2207636fce89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:06.052022 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:06.073006 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be7d2f0-9f03-4699-a364-407bf1f9247e None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:06.099834 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee5ee2-0ac4-4af7-a4f0-2207636fce89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:06.101968 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee5ee2-0ac4-4af7-a4f0-2207636fce89 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:06.109718 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dce99c-9f1c-4d22-bc18-5e6098e18653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:06.110573 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dce99c-9f1c-4d22-bc18-5e6098e18653 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:06.111046 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dce99c-9f1c-4d22-bc18-5e6098e18653 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:06.111598 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dce99c-9f1c-4d22-bc18-5e6098e18653 None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:06.112374 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dce99c-9f1c-4d22-bc18-5e6098e18653 None admin] RBAC: Adding query filter params (name=service) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:06.112742 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dce99c-9f1c-4d22-bc18-5e6098e18653 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:06.114565 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dce99c-9f1c-4d22-bc18-5e6098e18653 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:06.123634 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 280/553] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 17 23:31:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:06.124957 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd7761-e306-4b48-8344-1ebeea1d3e8e None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:06.125306 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd7761-e306-4b48-8344-1ebeea1d3e8e None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:06.125655 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd7761-e306-4b48-8344-1ebeea1d3e8e None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:06.130935 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 281/554] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:06.130935 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd7761-e306-4b48-8344-1ebeea1d3e8e None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:06.235146 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 274/555] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:31:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:31:06.236514 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637562e2-c3b7-4624-8ce1-463860057adb None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:06.236801 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637562e2-c3b7-4624-8ce1-463860057adb None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:06.237080 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637562e2-c3b7-4624-8ce1-463860057adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:06.241607 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd7761-e306-4b48-8344-1ebeea1d3e8e None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:06.255994 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-637562e2-c3b7-4624-8ce1-463860057adb None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:06.257513 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-637562e2-c3b7-4624-8ce1-463860057adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:06.275476 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:06.277988 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8f395-7ca2-48a2-9988-fe3cb38455ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:06.278619 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8f395-7ca2-48a2-9988-fe3cb38455ea None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:06.278895 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8f395-7ca2-48a2-9988-fe3cb38455ea None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:06.279165 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8f395-7ca2-48a2-9988-fe3cb38455ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:06.291634 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b8f395-7ca2-48a2-9988-fe3cb38455ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:06.293845 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b8f395-7ca2-48a2-9988-fe3cb38455ea None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:06.295736 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:06.310877 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-30b8f395-7ca2-48a2-9988-fe3cb38455ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:06.310877 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:06.310877 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:06.310877 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:06.310877 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:06.310877 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:06.310877 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:06.310877 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:06.310877 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:06.310877 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:06.310877 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:06.310877 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:06.310877 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:06.310877 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:06.310877 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:06.310877 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:06.311824 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:06.311824 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:06.311824 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:06.311824 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:06.311824 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:06.311824 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:06.311824 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:06.311824 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:06.311824 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:06.311824 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:06.311824 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:06.311824 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:06.311824 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:06.311824 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:06.311824 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:06.311824 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:06.311824 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:06.313051 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:06.313051 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:06.313051 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:06.313051 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 282/556] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:06.313051 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8f395-7ca2-48a2-9988-fe3cb38455ea None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:06.348088 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 275/557] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:31:06.349506 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637562e2-c3b7-4624-8ce1-463860057adb None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:06.424010 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8f395-7ca2-48a2-9988-fe3cb38455ea None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:06.460505 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fcf09b-18f8-4bf7-816a-77c4a11bdbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:06.461140 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fcf09b-18f8-4bf7-816a-77c4a11bdbe2 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:06.461414 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fcf09b-18f8-4bf7-816a-77c4a11bdbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:06.461681 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fcf09b-18f8-4bf7-816a-77c4a11bdbe2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:06.472943 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fcf09b-18f8-4bf7-816a-77c4a11bdbe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:06.474545 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fcf09b-18f8-4bf7-816a-77c4a11bdbe2 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:06.490770 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-53fcf09b-18f8-4bf7-816a-77c4a11bdbe2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:06.490770 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:06.490770 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:06.490770 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:06.490770 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:06.490770 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:06.490770 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:06.490770 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:06.490770 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:06.490770 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:06.490770 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:06.490770 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:06.490770 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:06.490770 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:06.490770 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:06.490770 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:06.492311 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:06.492311 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:06.492311 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:06.492311 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:06.492311 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:06.492311 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:06.492311 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:06.492311 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:06.492311 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:06.492311 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:06.492311 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:06.492311 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:06.492311 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:06.492311 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:06.492311 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:06.492311 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:06.492311 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:06.493207 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:06.493207 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:06.493207 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:06.493207 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 283/558] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:06.493689 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ee77a-bd83-43a6-bb1c-5009bd92b960 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:06.493792 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ee77a-bd83-43a6-bb1c-5009bd92b960 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:06.496057 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ee77a-bd83-43a6-bb1c-5009bd92b960 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:06.499456 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 284/559] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:06.500465 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ee77a-bd83-43a6-bb1c-5009bd92b960 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:06.543065 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637562e2-c3b7-4624-8ce1-463860057adb None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:06.578463 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:06.588091 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09230c85-080c-4833-a76a-eb531f99dde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:06.588829 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09230c85-080c-4833-a76a-eb531f99dde1 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:06.589277 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09230c85-080c-4833-a76a-eb531f99dde1 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:06.589945 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09230c85-080c-4833-a76a-eb531f99dde1 None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:06.590599 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09230c85-080c-4833-a76a-eb531f99dde1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:06.590900 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09230c85-080c-4833-a76a-eb531f99dde1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:06.592627 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09230c85-080c-4833-a76a-eb531f99dde1 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:06.606505 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 276/560] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:06.607744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c081eb3-b926-404b-98dc-deae7bcfa837 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:06.608056 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c081eb3-b926-404b-98dc-deae7bcfa837 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:06.608335 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c081eb3-b926-404b-98dc-deae7bcfa837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:06.627767 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ee77a-bd83-43a6-bb1c-5009bd92b960 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:06.628742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-9c081eb3-b926-404b-98dc-deae7bcfa837 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:06.630047 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-9c081eb3-b926-404b-98dc-deae7bcfa837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:06.633743 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:06.674563 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b1d0c3-2f8a-43d8-bc4d-83a743702aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:06.675222 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1d0c3-2f8a-43d8-bc4d-83a743702aac None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:06.675500 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1d0c3-2f8a-43d8-bc4d-83a743702aac None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:06.675776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1d0c3-2f8a-43d8-bc4d-83a743702aac None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:06.676494 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b1d0c3-2f8a-43d8-bc4d-83a743702aac None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:06.676804 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b1d0c3-2f8a-43d8-bc4d-83a743702aac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:06.678236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b1d0c3-2f8a-43d8-bc4d-83a743702aac None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:06.695526 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 285/561] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:06.696538 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b1d0c3-2f8a-43d8-bc4d-83a743702aac None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:06.719938 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 277/562] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:31:06.721091 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c081eb3-b926-404b-98dc-deae7bcfa837 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:06.812047 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b1d0c3-2f8a-43d8-bc4d-83a743702aac None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:06.821470 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:06.821700 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c081eb3-b926-404b-98dc-deae7bcfa837 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:06.852609 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914f8552-4420-4365-8a17-6d61a57ff3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:06.853682 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f8552-4420-4365-8a17-6d61a57ff3a4 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:06.854156 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f8552-4420-4365-8a17-6d61a57ff3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:06.855270 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f8552-4420-4365-8a17-6d61a57ff3a4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:06.857467 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0070a473-5446-463e-a91a-5cad21e2e192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:06.858078 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070a473-5446-463e-a91a-5cad21e2e192 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:06.858355 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070a473-5446-463e-a91a-5cad21e2e192 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:06.858625 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070a473-5446-463e-a91a-5cad21e2e192 None admin] PATH_INFO: `/v3/projects/service` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:06.864080 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914f8552-4420-4365-8a17-6d61a57ff3a4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:06.865433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0070a473-5446-463e-a91a-5cad21e2e192 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:06.865966 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914f8552-4420-4365-8a17-6d61a57ff3a4 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:06.866853 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0070a473-5446-463e-a91a-5cad21e2e192 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:06.869808 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:06.879505 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-0070a473-5446-463e-a91a-5cad21e2e192 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:06.879505 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:06.879505 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:06.879505 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:06.879505 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:06.879505 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:06.879505 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:06.879505 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:06.879505 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:06.879505 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:06.879505 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:06.879505 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:06.879505 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 23:31:06.879505 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 23:31:06.879505 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 23:31:06.880449 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 23:31:06.880449 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:06.880449 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:06.880449 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:06.880449 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:06.880449 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:06.880449 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:06.880449 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:06.880449 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:06.880449 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:06.880449 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:06.880449 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:06.880449 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:06.880449 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:06.880449 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:06.880449 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:06.880449 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:06.881413 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 23:31:06.881413 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 23:31:06.881413 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 23:31:06.881413 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 23:31:06.881413 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 23:31:06.881413 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 23:31:06.881413 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:06.881413 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:06.881850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-914f8552-4420-4365-8a17-6d61a57ff3a4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 17 23:31:06.881850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:06.881850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:06.881850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:06.881850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:06.881850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:06.881850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:06.881850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:06.881850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:06.881850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:06.881850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:06.881850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:06.881850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 23:31:06.881850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 23:31:06.881850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 23:31:06.881850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 23:31:06.882698 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:06.882698 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:06.882698 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 23:31:06.882698 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:06.882698 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 23:31:06.882698 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:06.882698 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:06.882698 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:06.882698 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:06.882698 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:06.882698 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:06.882698 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:06.882698 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:06.882698 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:06.882698 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:06.882698 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:06.882698 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:06.882698 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:06.883622 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:06.883622 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:06.883622 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 23:31:06.883622 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 23:31:06.883622 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 23:31:06.883622 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 23:31:06.883622 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 23:31:06.883622 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 23:31:06.883622 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 17 23:31:06.883622 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:06.883622 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 286/563] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:31:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:06.883622 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914f8552-4420-4365-8a17-6d61a57ff3a4 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:06.888530 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 278/564] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:31:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:06.889637 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0070a473-5446-463e-a91a-5cad21e2e192 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:06.992329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0070a473-5446-463e-a91a-5cad21e2e192 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:06.996749 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914f8552-4420-4365-8a17-6d61a57ff3a4 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:07.025888 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3831d7c-fcce-4c7b-ae10-e3831190510c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:07.027042 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3831d7c-fcce-4c7b-ae10-e3831190510c None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:07.027322 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3831d7c-fcce-4c7b-ae10-e3831190510c None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:07.027587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3831d7c-fcce-4c7b-ae10-e3831190510c None admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:07.028592 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3831d7c-fcce-4c7b-ae10-e3831190510c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:07.028891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3831d7c-fcce-4c7b-ae10-e3831190510c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:07.030309 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3831d7c-fcce-4c7b-ae10-e3831190510c None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:07.030538 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2298238f-a404-4335-9c22-49a1ad5d4c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:07.031157 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298238f-a404-4335-9c22-49a1ad5d4c93 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:07.031433 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298238f-a404-4335-9c22-49a1ad5d4c93 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:07.031715 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298238f-a404-4335-9c22-49a1ad5d4c93 None admin] PATH_INFO: `/v3/roles/service` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:07.041554 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2298238f-a404-4335-9c22-49a1ad5d4c93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:31:07.046649 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-2298238f-a404-4335-9c22-49a1ad5d4c93 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:31:07.046649 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:07.046649 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:07.046649 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:07.046649 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:07.046649 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:07.046649 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:07.046649 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:07.046649 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:07.046649 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:07.046649 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:07.046649 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:07.046649 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:31:07.046649 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:31:07.046649 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:31:07.046649 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:31:07.052601 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:31:07.052601 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:31:07.052601 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:31:07.052601 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:31:07.052601 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:07.052601 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:07.052601 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:07.052601 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:07.052601 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:07.052601 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:07.052601 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:07.052601 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:07.052601 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:07.052601 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:07.052601 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:07.052601 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:07.053451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:07.053451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:07.053451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:07.053451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:07.053451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:31:07.053451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:31:07.053451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:31:07.053451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:31:07.053451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:31:07.053451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:31:07.053451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:31:07.053451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:07.054361 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 279/565] 10.222.0.26 () {56 vars in 1137 bytes} [Wed Aug 17 23:31:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:07.054361 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32982143-71e2-4ea9-bc02-9805075f503d None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:07.054361 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32982143-71e2-4ea9-bc02-9805075f503d None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:07.054361 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32982143-71e2-4ea9-bc02-9805075f503d None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:07.054361 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 280/566] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:07.054361 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32982143-71e2-4ea9-bc02-9805075f503d None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:07.054765 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 287/567] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 17 23:31:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:07.054765 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2298238f-a404-4335-9c22-49a1ad5d4c93 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:07.080048 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:07.111296 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:07.151173 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2298238f-a404-4335-9c22-49a1ad5d4c93 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:07.157613 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32982143-71e2-4ea9-bc02-9805075f503d None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:07.184918 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f7c7d8-2294-47a7-921a-faeaad6947a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:07.185541 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7c7d8-2294-47a7-921a-faeaad6947a0 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:07.185826 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7c7d8-2294-47a7-921a-faeaad6947a0 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:07.186093 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7c7d8-2294-47a7-921a-faeaad6947a0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:07.190555 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625f459a-3e48-4e2c-89b0-8d08f1e4d163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:07.191136 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f459a-3e48-4e2c-89b0-8d08f1e4d163 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:07.191410 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f459a-3e48-4e2c-89b0-8d08f1e4d163 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:07.191676 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f459a-3e48-4e2c-89b0-8d08f1e4d163 None admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:07.192480 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625f459a-3e48-4e2c-89b0-8d08f1e4d163 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:07.192785 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625f459a-3e48-4e2c-89b0-8d08f1e4d163 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:07.194214 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625f459a-3e48-4e2c-89b0-8d08f1e4d163 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:07.195989 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f7c7d8-2294-47a7-921a-faeaad6947a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:31:07.201619 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-50f7c7d8-2294-47a7-921a-faeaad6947a0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:31:07.201619 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:07.201619 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:07.201619 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:07.201619 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:07.201619 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:07.201619 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:07.201619 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:07.201619 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:07.201619 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:07.201619 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:07.201619 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:07.201619 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:31:07.201619 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:31:07.201619 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:31:07.201619 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:31:07.202712 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:31:07.202712 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:31:07.202712 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:31:07.202712 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:31:07.202712 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:07.202712 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:07.202712 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:07.202712 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:07.202712 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:07.202712 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:07.202712 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:07.202712 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:07.202712 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:07.202712 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:07.202712 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:07.202712 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:07.203574 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:07.203574 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:07.203574 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:07.203574 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:07.203574 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:31:07.203574 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:31:07.203574 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:31:07.203574 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:31:07.203574 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:31:07.203574 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:31:07.203574 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:31:07.203574 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:07.203574 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 288/568] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 17 23:31:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:07.203574 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f7c7d8-2294-47a7-921a-faeaad6947a0 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:07.216613 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 281/569] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 17 23:31:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:07.216613 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625f459a-3e48-4e2c-89b0-8d08f1e4d163 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:07.286546 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:07.301644 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f7c7d8-2294-47a7-921a-faeaad6947a0 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:07.310262 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625f459a-3e48-4e2c-89b0-8d08f1e4d163 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:07.326740 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:07.335075 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc191c-8236-4439-b4a5-f6a933c11ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:07.335698 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc191c-8236-4439-b4a5-f6a933c11ad6 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:07.335990 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc191c-8236-4439-b4a5-f6a933c11ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:07.336281 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc191c-8236-4439-b4a5-f6a933c11ad6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:07.343814 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fafc2d1-2535-4f81-9014-92449522f431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:07.344436 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafc2d1-2535-4f81-9014-92449522f431 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:07.344711 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafc2d1-2535-4f81-9014-92449522f431 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:07.344977 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafc2d1-2535-4f81-9014-92449522f431 None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:07.345636 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fafc2d1-2535-4f81-9014-92449522f431 None admin] RBAC: Adding query filter params (name=service) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:07.345931 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fafc2d1-2535-4f81-9014-92449522f431 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:07.347020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cc191c-8236-4439-b4a5-f6a933c11ad6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:07.347298 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fafc2d1-2535-4f81-9014-92449522f431 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:07.348488 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cc191c-8236-4439-b4a5-f6a933c11ad6 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:07.355306 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 282/570] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 17 23:31:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:07.356454 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd044ae9-d217-4f42-a28d-f736fbc5be4f None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:07.356734 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd044ae9-d217-4f42-a28d-f736fbc5be4f None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:07.357004 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd044ae9-d217-4f42-a28d-f736fbc5be4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:07.366155 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-e7cc191c-8236-4439-b4a5-f6a933c11ad6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:07.366155 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:07.366155 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:07.366155 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:07.366155 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:07.366155 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:07.366155 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:07.366155 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:07.366155 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:07.366155 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:07.366155 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:07.366155 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:07.366155 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:07.366155 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:07.366155 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:07.366155 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:07.367087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:07.367087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:07.367087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:07.367087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:07.367087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:07.367087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:07.367087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:07.367087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:07.367087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:07.367087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:07.367087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:07.367087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:07.367087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:07.367087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:07.367087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:07.367087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:07.367087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:07.368018 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:07.368018 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:07.368018 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:07.368018 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 289/571] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:07.368018 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc191c-8236-4439-b4a5-f6a933c11ad6 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:07.374343 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-fd044ae9-d217-4f42-a28d-f736fbc5be4f None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:07.375525 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-fd044ae9-d217-4f42-a28d-f736fbc5be4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:07.442244 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 283/572] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:31:07.443364 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd044ae9-d217-4f42-a28d-f736fbc5be4f None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:07.477865 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc191c-8236-4439-b4a5-f6a933c11ad6 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:07.513441 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0af2a3-a6bf-467c-bad2-78552b3b8efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:07.514038 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0af2a3-a6bf-467c-bad2-78552b3b8efd None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:07.514314 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0af2a3-a6bf-467c-bad2-78552b3b8efd None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:07.514580 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0af2a3-a6bf-467c-bad2-78552b3b8efd None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:07.515241 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0af2a3-a6bf-467c-bad2-78552b3b8efd None admin] RBAC: Adding query filter params (name=service) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:07.515538 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0af2a3-a6bf-467c-bad2-78552b3b8efd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:07.516925 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0af2a3-a6bf-467c-bad2-78552b3b8efd None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:07.522438 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:07.526571 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 290/573] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 17 23:31:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:07.527611 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0af2a3-a6bf-467c-bad2-78552b3b8efd None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:07.552068 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd044ae9-d217-4f42-a28d-f736fbc5be4f None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:07.563755 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:07.589340 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce749e-8293-4e3e-9327-bc96f66d7df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:07.589942 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce749e-8293-4e3e-9327-bc96f66d7df5 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:07.590215 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce749e-8293-4e3e-9327-bc96f66d7df5 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:07.590482 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce749e-8293-4e3e-9327-bc96f66d7df5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:07.591193 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adce749e-8293-4e3e-9327-bc96f66d7df5 None admin] RBAC: Adding query filter params (role.id=b64f80e6f3ea4ad49b94acdde42ccd87, scope.project.id=4de86915ef4340248ba8a8f51d56594e, user.id=ee422ea90557444fabdc9c292865b9b8) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:07.591496 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adce749e-8293-4e3e-9327-bc96f66d7df5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:07.592950 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adce749e-8293-4e3e-9327-bc96f66d7df5 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:07.598422 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 284/574] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:31:07 2022] GET /identity/v3/role_assignments?user.id=ee422ea90557444fabdc9c292865b9b8&scope.project.id=4de86915ef4340248ba8a8f51d56594e&role.id=b64f80e6f3ea4ad49b94acdde42ccd87 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:07.599181 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce749e-8293-4e3e-9327-bc96f66d7df5 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:07.655035 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0af2a3-a6bf-467c-bad2-78552b3b8efd None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:07.692699 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a51fe-b153-461e-bbe7-65348d0dc920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:07.693335 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a51fe-b153-461e-bbe7-65348d0dc920 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:07.693609 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a51fe-b153-461e-bbe7-65348d0dc920 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:07.693877 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a51fe-b153-461e-bbe7-65348d0dc920 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:07.709631 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970a51fe-b153-461e-bbe7-65348d0dc920 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:07.711106 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970a51fe-b153-461e-bbe7-65348d0dc920 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:07.727561 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-970a51fe-b153-461e-bbe7-65348d0dc920 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:07.727561 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:07.727561 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:07.727561 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:07.727561 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:07.727561 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:07.727561 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:07.727561 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:07.727561 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:07.727561 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:07.727561 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:07.727561 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:07.727561 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:07.727561 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:07.727561 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:07.727561 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:07.728781 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:07.728781 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:07.728781 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:07.728781 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:07.728781 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:07.728781 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:07.728781 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:07.728781 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:07.728781 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:07.728781 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:07.728781 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:07.728781 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:07.728781 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:07.728781 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:07.728781 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:07.728781 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:07.728781 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:07.730425 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:07.730425 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:07.730425 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:07.730425 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 291/575] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:07.730425 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a51fe-b153-461e-bbe7-65348d0dc920 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:07.757398 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce749e-8293-4e3e-9327-bc96f66d7df5 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:07.789634 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:07.796202 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9c5f88-dbbf-43e9-affd-d47787bd5a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:07.796719 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9c5f88-dbbf-43e9-affd-d47787bd5a6a None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:07.796997 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9c5f88-dbbf-43e9-affd-d47787bd5a6a None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:07.797272 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9c5f88-dbbf-43e9-affd-d47787bd5a6a None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:07.797897 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9c5f88-dbbf-43e9-affd-d47787bd5a6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:07.798198 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9c5f88-dbbf-43e9-affd-d47787bd5a6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:07.799565 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9c5f88-dbbf-43e9-affd-d47787bd5a6a None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:07.812480 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 285/576] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:07.813608 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9c5f88-dbbf-43e9-affd-d47787bd5a6a None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:07.846446 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a51fe-b153-461e-bbe7-65348d0dc920 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:07.849472 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:07.881529 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03c7cf-7bcb-40b5-b2a2-44415a0df337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:07.882161 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03c7cf-7bcb-40b5-b2a2-44415a0df337 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:07.882440 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03c7cf-7bcb-40b5-b2a2-44415a0df337 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:07.882705 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03c7cf-7bcb-40b5-b2a2-44415a0df337 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:07.894823 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df03c7cf-7bcb-40b5-b2a2-44415a0df337 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:07.896439 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df03c7cf-7bcb-40b5-b2a2-44415a0df337 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:07.912247 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-df03c7cf-7bcb-40b5-b2a2-44415a0df337 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:07.912247 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:07.912247 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:07.912247 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:07.912247 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:07.912247 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:07.912247 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:07.912247 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:07.912247 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:07.912247 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:07.912247 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:07.912247 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:07.912247 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:07.912247 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:07.912247 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:07.912247 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:07.913173 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:07.913173 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:07.913173 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:07.913173 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:07.913173 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:07.913173 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:07.913173 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:07.913173 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:07.913173 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:07.913173 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:07.913173 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:07.913173 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:07.913173 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:07.913173 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:07.913173 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:07.913173 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:07.913173 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:07.914055 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:07.914055 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:07.914055 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:07.914055 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 292/577] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:07.914055 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03c7cf-7bcb-40b5-b2a2-44415a0df337 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:07.925818 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9c5f88-dbbf-43e9-affd-d47787bd5a6a None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:07.959599 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84183263-18ca-4e83-9bc2-dc5cf9bfc67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:07.960253 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84183263-18ca-4e83-9bc2-dc5cf9bfc67d None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:07.960538 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84183263-18ca-4e83-9bc2-dc5cf9bfc67d None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:07.960813 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84183263-18ca-4e83-9bc2-dc5cf9bfc67d None admin] PATH_INFO: `/v3/roles/service` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:07.971511 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84183263-18ca-4e83-9bc2-dc5cf9bfc67d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:31:07.976384 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-84183263-18ca-4e83-9bc2-dc5cf9bfc67d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:31:07.976384 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:07.976384 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:07.976384 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:07.976384 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:07.976384 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:07.976384 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:07.976384 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:07.976384 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:07.976384 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:07.976384 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:07.976384 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:07.976384 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:31:07.976384 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:31:07.976384 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:31:07.976384 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:31:07.977487 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:31:07.977487 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:31:07.977487 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:31:07.977487 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:31:07.977487 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:07.977487 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:07.977487 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:07.977487 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:07.977487 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:07.977487 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:07.977487 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:07.977487 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:07.977487 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:07.977487 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:07.977487 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:07.977487 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:07.978371 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:07.978371 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:07.978371 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:07.978371 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:07.978371 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:31:07.978371 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:31:07.978371 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:31:07.978371 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:31:07.978371 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:31:07.978371 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:31:07.978371 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:31:07.978371 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:07.978371 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 286/578] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 17 23:31:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:07.979049 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84183263-18ca-4e83-9bc2-dc5cf9bfc67d None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:08.022031 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03c7cf-7bcb-40b5-b2a2-44415a0df337 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:08.024839 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:08.060296 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f35cdf5-4167-49e2-ae52-32cc2bd4832c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:08.060910 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35cdf5-4167-49e2-ae52-32cc2bd4832c None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:08.061194 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35cdf5-4167-49e2-ae52-32cc2bd4832c None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:08.061467 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35cdf5-4167-49e2-ae52-32cc2bd4832c None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:08.062101 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f35cdf5-4167-49e2-ae52-32cc2bd4832c None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:08.062399 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f35cdf5-4167-49e2-ae52-32cc2bd4832c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:08.063758 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f35cdf5-4167-49e2-ae52-32cc2bd4832c None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:08.076163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 293/579] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:08.076968 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f35cdf5-4167-49e2-ae52-32cc2bd4832c None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:08.083812 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:08.093831 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84183263-18ca-4e83-9bc2-dc5cf9bfc67d None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:08.129139 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7bc01d-4d4f-402c-a4e9-56f65b0d48c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:08.129792 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7bc01d-4d4f-402c-a4e9-56f65b0d48c3 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:08.130075 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7bc01d-4d4f-402c-a4e9-56f65b0d48c3 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:08.130350 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7bc01d-4d4f-402c-a4e9-56f65b0d48c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:08.138483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7bc01d-4d4f-402c-a4e9-56f65b0d48c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:08.139947 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7bc01d-4d4f-402c-a4e9-56f65b0d48c3 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:08.151061 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-9f7bc01d-4d4f-402c-a4e9-56f65b0d48c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:08.151061 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:08.151061 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:08.151061 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:08.151061 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:08.151061 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:08.151061 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:08.151061 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:08.151061 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:08.151061 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:08.151061 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:08.151061 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:08.151061 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 23:31:08.151061 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 23:31:08.151061 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 23:31:08.152020 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 23:31:08.152020 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:08.152020 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:08.152020 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:08.152020 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:08.152020 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:08.152020 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:08.152020 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:08.152020 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:08.152020 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:08.152020 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:08.152020 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:08.152020 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:08.152020 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:08.152020 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:08.152020 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:08.152020 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:08.153207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 23:31:08.153207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 23:31:08.153207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 23:31:08.153207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 23:31:08.153207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 23:31:08.153207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 23:31:08.153207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:08.153207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:08.157157 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 287/580] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:31:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:08.157773 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7bc01d-4d4f-402c-a4e9-56f65b0d48c3 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:08.177239 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f35cdf5-4167-49e2-ae52-32cc2bd4832c None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:08.214261 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c66f196-5ce0-4cd0-aaa3-4164d4db096b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:08.214965 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66f196-5ce0-4cd0-aaa3-4164d4db096b None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:08.215322 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66f196-5ce0-4cd0-aaa3-4164d4db096b None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:08.215670 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66f196-5ce0-4cd0-aaa3-4164d4db096b None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:08.216411 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c66f196-5ce0-4cd0-aaa3-4164d4db096b None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:08.216716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c66f196-5ce0-4cd0-aaa3-4164d4db096b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:08.218127 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c66f196-5ce0-4cd0-aaa3-4164d4db096b None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:08.230033 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 294/581] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:08.230908 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c66f196-5ce0-4cd0-aaa3-4164d4db096b None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:08.257972 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:08.270373 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7bc01d-4d4f-402c-a4e9-56f65b0d48c3 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:08.304438 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5175d0-ebeb-438e-a8b5-a28c17a8ef87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:08.305031 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5175d0-ebeb-438e-a8b5-a28c17a8ef87 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:08.305300 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5175d0-ebeb-438e-a8b5-a28c17a8ef87 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:08.305576 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5175d0-ebeb-438e-a8b5-a28c17a8ef87 None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:08.306242 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5175d0-ebeb-438e-a8b5-a28c17a8ef87 None admin] RBAC: Adding query filter params (name=service) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:08.306543 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5175d0-ebeb-438e-a8b5-a28c17a8ef87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:08.308452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5175d0-ebeb-438e-a8b5-a28c17a8ef87 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:08.316429 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 288/582] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 17 23:31:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:08.320846 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5175d0-ebeb-438e-a8b5-a28c17a8ef87 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:08.336835 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:08.342148 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c66f196-5ce0-4cd0-aaa3-4164d4db096b None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:08.378637 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e89394-87aa-461f-a789-a1a8dcb626c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:08.379264 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e89394-87aa-461f-a789-a1a8dcb626c7 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:08.379544 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e89394-87aa-461f-a789-a1a8dcb626c7 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:08.379813 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e89394-87aa-461f-a789-a1a8dcb626c7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:08.387284 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e89394-87aa-461f-a789-a1a8dcb626c7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:08.388766 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e89394-87aa-461f-a789-a1a8dcb626c7 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:08.400402 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-09e89394-87aa-461f-a789-a1a8dcb626c7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 17 23:31:08.400402 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:08.400402 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:08.400402 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:08.400402 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:08.400402 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:08.400402 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:08.400402 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:08.400402 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:08.400402 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:08.400402 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:08.400402 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:08.400402 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 23:31:08.400402 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 23:31:08.400402 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 23:31:08.400402 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 23:31:08.401876 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:08.401876 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:08.401876 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 23:31:08.401876 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:08.401876 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 23:31:08.401876 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:08.401876 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:08.401876 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:08.401876 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:08.401876 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:08.401876 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:08.401876 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:08.401876 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:08.401876 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:08.401876 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:08.401876 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:08.401876 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:08.401876 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:08.403133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:08.403133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:08.403133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 23:31:08.403133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 23:31:08.403133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 23:31:08.403133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 23:31:08.403133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 23:31:08.403133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 23:31:08.403133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 17 23:31:08.403133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:08.403133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 295/583] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:31:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:08.403133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e89394-87aa-461f-a789-a1a8dcb626c7 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:08.426501 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5175d0-ebeb-438e-a8b5-a28c17a8ef87 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:08.461035 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781fd47e-d20a-45f9-ab18-e8b983034ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:08.461644 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781fd47e-d20a-45f9-ab18-e8b983034ee8 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:08.461925 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781fd47e-d20a-45f9-ab18-e8b983034ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:08.462199 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781fd47e-d20a-45f9-ab18-e8b983034ee8 None admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:08.462894 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781fd47e-d20a-45f9-ab18-e8b983034ee8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:08.463200 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781fd47e-d20a-45f9-ab18-e8b983034ee8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:08.465190 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781fd47e-d20a-45f9-ab18-e8b983034ee8 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:08.477635 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 289/584] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 17 23:31:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:08.478491 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781fd47e-d20a-45f9-ab18-e8b983034ee8 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:08.515431 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:08.535896 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e89394-87aa-461f-a789-a1a8dcb626c7 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:08.561255 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:08.575541 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da6475-c064-48a2-ad95-60f653b74575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:08.576200 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da6475-c064-48a2-ad95-60f653b74575 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:08.576477 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da6475-c064-48a2-ad95-60f653b74575 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:08.576747 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da6475-c064-48a2-ad95-60f653b74575 None admin] PATH_INFO: `/v3/users/nova` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:08.584988 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0da6475-c064-48a2-ad95-60f653b74575 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:08.586423 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0da6475-c064-48a2-ad95-60f653b74575 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:08.586845 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781fd47e-d20a-45f9-ab18-e8b983034ee8 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:08.598678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-c0da6475-c064-48a2-ad95-60f653b74575 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 17 23:31:08.598678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:08.598678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:08.598678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:08.598678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:08.598678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:08.598678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:08.598678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:08.598678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:08.598678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:08.598678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:08.598678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:08.598678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 23:31:08.598678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 23:31:08.598678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 23:31:08.598678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 23:31:08.604358 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:08.604358 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:08.604358 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 23:31:08.604358 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:08.604358 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 23:31:08.604358 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:08.604358 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:08.604358 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:08.604358 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:08.604358 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:08.604358 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:08.604358 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:08.604358 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:08.604358 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:08.604358 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:08.604358 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:08.604358 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:08.604358 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:08.606131 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:08.606131 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:08.606131 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 23:31:08.606131 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 23:31:08.606131 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 23:31:08.606131 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 23:31:08.606131 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 23:31:08.606131 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 23:31:08.606131 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 17 23:31:08.606131 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:08.606131 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 296/585] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Aug 17 23:31:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:08.606131 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da6475-c064-48a2-ad95-60f653b74575 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:08.630607 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ba4745-018a-4188-9537-28f516deab57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:08.631236 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba4745-018a-4188-9537-28f516deab57 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:08.631616 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba4745-018a-4188-9537-28f516deab57 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:08.631792 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba4745-018a-4188-9537-28f516deab57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:08.642771 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ba4745-018a-4188-9537-28f516deab57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:08.644271 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ba4745-018a-4188-9537-28f516deab57 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:08.659849 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-97ba4745-018a-4188-9537-28f516deab57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:08.659849 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:08.659849 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:08.659849 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:08.659849 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:08.659849 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:08.659849 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:08.659849 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:08.659849 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:08.659849 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:08.659849 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:08.659849 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:08.659849 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:08.659849 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:08.659849 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:08.659849 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:08.661084 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:08.661084 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:08.661084 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:08.661084 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:08.661084 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:08.661084 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:08.661084 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:08.661084 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:08.661084 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:08.661084 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:08.661084 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:08.661084 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:08.661084 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:08.661084 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:08.661084 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:08.661084 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:08.661084 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:08.662147 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:08.662147 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:08.662147 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:08.662147 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 290/586] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:08.666647 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ba4745-018a-4188-9537-28f516deab57 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:08.720698 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da6475-c064-48a2-ad95-60f653b74575 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:08.756794 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c83f0-ed20-48e6-a85a-99a46ed7d7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:08.757406 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c83f0-ed20-48e6-a85a-99a46ed7d7dc None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:08.757690 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c83f0-ed20-48e6-a85a-99a46ed7d7dc None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:08.757989 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c83f0-ed20-48e6-a85a-99a46ed7d7dc None admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:08.758690 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8c83f0-ed20-48e6-a85a-99a46ed7d7dc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:08.758995 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8c83f0-ed20-48e6-a85a-99a46ed7d7dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:08.760713 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8c83f0-ed20-48e6-a85a-99a46ed7d7dc None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:08.769318 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ba4745-018a-4188-9537-28f516deab57 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:08.783191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 297/587] 10.222.0.26 () {56 vars in 1137 bytes} [Wed Aug 17 23:31:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:08.783906 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c83f0-ed20-48e6-a85a-99a46ed7d7dc None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:08.786726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:08.797394 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:08.807090 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3449fc-13e4-48f2-9f78-c1c60a142a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:08.808324 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3449fc-13e4-48f2-9f78-c1c60a142a8b None admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:08.808608 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3449fc-13e4-48f2-9f78-c1c60a142a8b None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:08.809303 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3449fc-13e4-48f2-9f78-c1c60a142a8b None admin] PATH_INFO: `/v3/projects/4de86915ef4340248ba8a8f51d56594e/users/9c08c497a14c426f9cfe0dd8c8100a21/roles/b64f80e6f3ea4ad49b94acdde42ccd87` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:08.824307 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3449fc-13e4-48f2-9f78-c1c60a142a8b None admin] RBAC: Authorizing `identity:create_grant(project_id=4de86915ef4340248ba8a8f51d56594e, user_id=9c08c497a14c426f9cfe0dd8c8100a21, role_id=b64f80e6f3ea4ad49b94acdde42ccd87)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:08.825762 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3449fc-13e4-48f2-9f78-c1c60a142a8b None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:08.838745 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 291/588] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 17 23:31:08 2022] PUT /identity/v3/projects/4de86915ef4340248ba8a8f51d56594e/users/9c08c497a14c426f9cfe0dd8c8100a21/roles/b64f80e6f3ea4ad49b94acdde42ccd87 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:31:08.845019 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3449fc-13e4-48f2-9f78-c1c60a142a8b None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:08.897760 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c83f0-ed20-48e6-a85a-99a46ed7d7dc None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:08.939849 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcc248-3f12-4917-954f-662b0cc8f8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:08.940802 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcc248-3f12-4917-954f-662b0cc8f8d4 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:08.941160 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcc248-3f12-4917-954f-662b0cc8f8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:08.941611 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcc248-3f12-4917-954f-662b0cc8f8d4 None admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:08.942428 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fcc248-3f12-4917-954f-662b0cc8f8d4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:08.942832 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fcc248-3f12-4917-954f-662b0cc8f8d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:08.944646 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fcc248-3f12-4917-954f-662b0cc8f8d4 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:08.971233 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 298/589] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Aug 17 23:31:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:08.972368 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcc248-3f12-4917-954f-662b0cc8f8d4 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:08.995712 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3449fc-13e4-48f2-9f78-c1c60a142a8b None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:09.038353 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0598a3d3-a47c-4479-89bf-4ae71bf8ae36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:09.039333 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598a3d3-a47c-4479-89bf-4ae71bf8ae36 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:09.039760 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598a3d3-a47c-4479-89bf-4ae71bf8ae36 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:09.040324 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598a3d3-a47c-4479-89bf-4ae71bf8ae36 None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:09.041110 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0598a3d3-a47c-4479-89bf-4ae71bf8ae36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:09.041548 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0598a3d3-a47c-4479-89bf-4ae71bf8ae36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:09.043262 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0598a3d3-a47c-4479-89bf-4ae71bf8ae36 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:09.043884 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:09.064290 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 292/590] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:09.065095 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0598a3d3-a47c-4479-89bf-4ae71bf8ae36 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:09.092155 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcc248-3f12-4917-954f-662b0cc8f8d4 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:09.116923 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:09.150871 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae50cc9f-2bba-49a0-b0a1-e81fc35df785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:09.151909 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50cc9f-2bba-49a0-b0a1-e81fc35df785 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:09.152927 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50cc9f-2bba-49a0-b0a1-e81fc35df785 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:09.153855 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50cc9f-2bba-49a0-b0a1-e81fc35df785 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:09.173474 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae50cc9f-2bba-49a0-b0a1-e81fc35df785 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:09.175187 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae50cc9f-2bba-49a0-b0a1-e81fc35df785 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:09.205848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-ae50cc9f-2bba-49a0-b0a1-e81fc35df785 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:09.205848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:09.205848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:09.205848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:09.205848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:09.205848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:09.205848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:09.205848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:09.205848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:09.205848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:09.205848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:09.205848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:09.205848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:09.205848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:09.205848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:09.205848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:09.207041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:09.207041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:09.207041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:09.207041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:09.207041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:09.207041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:09.207041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:09.207041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:09.207041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:09.207041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:09.207041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:09.207041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:09.207041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:09.207041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:09.207041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:09.207041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:09.207041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:09.208159 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:09.208159 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:09.208159 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:09.211573 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 299/591] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:09.211573 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae50cc9f-2bba-49a0-b0a1-e81fc35df785 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:09.243665 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0598a3d3-a47c-4479-89bf-4ae71bf8ae36 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:09.322175 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79da06-e42a-4812-8d07-fa90aeafacc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:09.325946 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79da06-e42a-4812-8d07-fa90aeafacc1 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:09.325946 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79da06-e42a-4812-8d07-fa90aeafacc1 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:09.325946 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79da06-e42a-4812-8d07-fa90aeafacc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:09.339988 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae50cc9f-2bba-49a0-b0a1-e81fc35df785 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:09.353627 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:09.355807 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa79da06-e42a-4812-8d07-fa90aeafacc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:09.358362 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa79da06-e42a-4812-8d07-fa90aeafacc1 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:09.386809 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-aa79da06-e42a-4812-8d07-fa90aeafacc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:09.386809 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:09.386809 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:09.386809 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:09.386809 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:09.386809 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:09.386809 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:09.386809 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:09.386809 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:09.386809 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:09.386809 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:09.386809 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:09.386809 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:09.386809 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:09.386809 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:09.386809 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:09.388068 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:09.388068 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:09.388068 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:09.388068 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:09.388068 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:09.388068 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:09.388068 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:09.388068 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:09.388068 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:09.388068 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:09.388068 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:09.388068 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:09.388068 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:09.388068 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:09.388068 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:09.388068 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:09.388068 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:09.389193 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:09.389193 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:09.389193 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:09.389746 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f75c7b2-db6b-4459-94c7-ebfade8da36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:09.390541 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75c7b2-db6b-4459-94c7-ebfade8da36f None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:09.390710 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 293/592] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:09.391107 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75c7b2-db6b-4459-94c7-ebfade8da36f None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:09.391538 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75c7b2-db6b-4459-94c7-ebfade8da36f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:09.391811 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79da06-e42a-4812-8d07-fa90aeafacc1 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:09.400080 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f75c7b2-db6b-4459-94c7-ebfade8da36f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:09.402017 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f75c7b2-db6b-4459-94c7-ebfade8da36f None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:09.437863 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-9f75c7b2-db6b-4459-94c7-ebfade8da36f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 17 23:31:09.437863 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:09.437863 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:09.437863 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:09.437863 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:09.437863 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:09.437863 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:09.437863 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:09.437863 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:09.437863 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:09.437863 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:09.437863 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:09.437863 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 23:31:09.437863 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 23:31:09.437863 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 23:31:09.437863 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 23:31:09.438867 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:09.438867 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:09.438867 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 23:31:09.438867 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:09.438867 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 23:31:09.438867 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:09.438867 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:09.438867 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:09.438867 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:09.438867 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:09.438867 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:09.438867 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:09.438867 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:09.438867 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:09.438867 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:09.438867 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:09.438867 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:09.438867 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:09.440035 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:09.440035 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:09.440035 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 23:31:09.440035 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 23:31:09.440035 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 23:31:09.440035 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 23:31:09.440035 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 23:31:09.440035 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 23:31:09.440035 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 17 23:31:09.440035 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:09.447057 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 300/593] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 17 23:31:09 2022] GET /identity/v3/users/neutron => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:09.448311 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f75c7b2-db6b-4459-94c7-ebfade8da36f None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:09.515432 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:09.527046 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79da06-e42a-4812-8d07-fa90aeafacc1 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:09.574353 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410083fe-e6c5-4116-8c05-1867246ab12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:09.575217 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410083fe-e6c5-4116-8c05-1867246ab12b None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:09.576102 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410083fe-e6c5-4116-8c05-1867246ab12b None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:09.576507 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410083fe-e6c5-4116-8c05-1867246ab12b None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:09.577737 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410083fe-e6c5-4116-8c05-1867246ab12b None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:09.578141 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410083fe-e6c5-4116-8c05-1867246ab12b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:09.583410 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410083fe-e6c5-4116-8c05-1867246ab12b None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:09.596252 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 294/594] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:09.597233 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410083fe-e6c5-4116-8c05-1867246ab12b None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:09.608699 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f75c7b2-db6b-4459-94c7-ebfade8da36f None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:09.644720 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b4df66-e784-4151-b696-3d98a7317d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:09.645596 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4df66-e784-4151-b696-3d98a7317d92 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:09.646056 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4df66-e784-4151-b696-3d98a7317d92 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:09.646638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4df66-e784-4151-b696-3d98a7317d92 None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:09.647570 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b4df66-e784-4151-b696-3d98a7317d92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:09.648084 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b4df66-e784-4151-b696-3d98a7317d92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:09.649623 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b4df66-e784-4151-b696-3d98a7317d92 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:09.662159 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 301/595] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:09.663392 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b45b4-32fe-4dd3-b351-f8cfe0c60725 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:09.663680 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b45b4-32fe-4dd3-b351-f8cfe0c60725 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:09.663980 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b45b4-32fe-4dd3-b351-f8cfe0c60725 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:09.665316 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 302/596] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:09.670641 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b45b4-32fe-4dd3-b351-f8cfe0c60725 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:09.700471 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:09.707334 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410083fe-e6c5-4116-8c05-1867246ab12b None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:09.742736 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ece09b2-225f-4470-ab7c-3645fa6f5401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:09.743405 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece09b2-225f-4470-ab7c-3645fa6f5401 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:09.743685 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece09b2-225f-4470-ab7c-3645fa6f5401 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:09.743950 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece09b2-225f-4470-ab7c-3645fa6f5401 None admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:09.744709 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ece09b2-225f-4470-ab7c-3645fa6f5401 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:09.745011 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ece09b2-225f-4470-ab7c-3645fa6f5401 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:09.746427 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ece09b2-225f-4470-ab7c-3645fa6f5401 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:09.767647 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 295/597] 10.222.0.26 () {56 vars in 1139 bytes} [Wed Aug 17 23:31:09 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:09.767647 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ece09b2-225f-4470-ab7c-3645fa6f5401 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:09.778563 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:09.782429 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b45b4-32fe-4dd3-b351-f8cfe0c60725 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:09.818300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32fe032-1f02-4fbc-b48f-7a85830c367e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:09.818923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32fe032-1f02-4fbc-b48f-7a85830c367e None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:09.819198 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32fe032-1f02-4fbc-b48f-7a85830c367e None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:09.819496 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32fe032-1f02-4fbc-b48f-7a85830c367e None admin] PATH_INFO: `/v3/projects/service` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:09.826682 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32fe032-1f02-4fbc-b48f-7a85830c367e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:09.828136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32fe032-1f02-4fbc-b48f-7a85830c367e None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:09.839501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-d32fe032-1f02-4fbc-b48f-7a85830c367e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:09.839501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:09.839501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:09.839501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:09.839501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:09.839501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:09.839501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:09.839501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:09.839501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:09.839501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:09.839501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:09.839501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:09.839501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 23:31:09.839501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 23:31:09.839501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 23:31:09.840608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 23:31:09.840608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:09.840608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:09.840608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:09.840608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:09.840608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:09.840608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:09.840608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:09.840608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:09.840608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:09.840608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:09.840608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:09.840608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:09.840608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:09.840608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:09.840608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:09.840608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:09.841484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 23:31:09.841484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 23:31:09.841484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 23:31:09.841484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 23:31:09.841484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 23:31:09.841484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 23:31:09.841484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:09.841484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:09.841484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 303/598] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:31:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:09.842812 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0932eb-d7d1-4677-a749-72a343fd8fe4 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:09.842812 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0932eb-d7d1-4677-a749-72a343fd8fe4 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:09.842812 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0932eb-d7d1-4677-a749-72a343fd8fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:09.860134 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-9b0932eb-d7d1-4677-a749-72a343fd8fe4 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:09.861311 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-9b0932eb-d7d1-4677-a749-72a343fd8fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:09.880518 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ece09b2-225f-4470-ab7c-3645fa6f5401 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:09.918743 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c50646-b819-4dd3-80e1-e11c97b6808d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:09.919649 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c50646-b819-4dd3-80e1-e11c97b6808d None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:09.919932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c50646-b819-4dd3-80e1-e11c97b6808d None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:09.920230 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c50646-b819-4dd3-80e1-e11c97b6808d None admin] PATH_INFO: `/v3/projects/service` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:09.929176 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c50646-b819-4dd3-80e1-e11c97b6808d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:09.930628 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c50646-b819-4dd3-80e1-e11c97b6808d None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:09.940170 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 304/599] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:31:09.940170 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0932eb-d7d1-4677-a749-72a343fd8fe4 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:09.953895 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-65c50646-b819-4dd3-80e1-e11c97b6808d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:09.953895 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:09.953895 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:09.953895 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:09.953895 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:09.953895 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:09.953895 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:09.953895 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:09.953895 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:09.953895 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:09.953895 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:09.953895 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:09.953895 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 23:31:09.953895 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 23:31:09.953895 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 23:31:09.954844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 23:31:09.954844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:09.954844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:09.954844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:09.954844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:09.954844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:09.954844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:09.954844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:09.954844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:09.954844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:09.954844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:09.954844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:09.954844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:09.954844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:09.954844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:09.954844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:09.954844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:09.955717 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 23:31:09.955717 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 23:31:09.955717 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 23:31:09.955717 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 23:31:09.955717 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 23:31:09.955717 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 23:31:09.955717 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:09.955717 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:09.962342 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 296/600] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:31:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:09.962342 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c50646-b819-4dd3-80e1-e11c97b6808d None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:09.965017 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:10.047314 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:10.048071 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0932eb-d7d1-4677-a749-72a343fd8fe4 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:10.084815 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c50646-b819-4dd3-80e1-e11c97b6808d None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:10.094164 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65873068-e391-48ab-848a-91773cc5b43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:10.095422 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65873068-e391-48ab-848a-91773cc5b43d None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:10.095917 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65873068-e391-48ab-848a-91773cc5b43d None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:10.096408 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65873068-e391-48ab-848a-91773cc5b43d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:10.109955 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65873068-e391-48ab-848a-91773cc5b43d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:10.111987 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65873068-e391-48ab-848a-91773cc5b43d None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:10.125727 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d84fc5-f6da-4965-b03c-63b4ab14a67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:10.126441 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d84fc5-f6da-4965-b03c-63b4ab14a67e None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:10.126824 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d84fc5-f6da-4965-b03c-63b4ab14a67e None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:10.127216 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d84fc5-f6da-4965-b03c-63b4ab14a67e None admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:10.128040 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d84fc5-f6da-4965-b03c-63b4ab14a67e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:10.128448 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d84fc5-f6da-4965-b03c-63b4ab14a67e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:10.130033 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d84fc5-f6da-4965-b03c-63b4ab14a67e None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:10.131254 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-65873068-e391-48ab-848a-91773cc5b43d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:10.131254 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:10.131254 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:10.131254 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:10.131254 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:10.131254 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:10.131254 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:10.131254 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:10.131254 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:10.131254 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:10.131254 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:10.131254 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:10.131254 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:10.131254 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:10.131254 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:10.131254 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:10.132218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:10.132218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:10.132218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:10.132218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:10.132218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:10.132218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:10.132218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:10.132218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:10.132218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:10.132218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:10.132218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:10.132218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:10.132218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:10.132218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:10.132218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:10.132218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:10.132218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:10.133108 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:10.133108 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:10.133108 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:10.135400 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 305/601] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:10.136410 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65873068-e391-48ab-848a-91773cc5b43d None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:10.146710 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 297/602] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 17 23:31:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:10.148781 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d84fc5-f6da-4965-b03c-63b4ab14a67e None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:10.233212 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:10.264143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65873068-e391-48ab-848a-91773cc5b43d None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:10.270546 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d84fc5-f6da-4965-b03c-63b4ab14a67e None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:10.300060 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:10.300353 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db47466f-e822-4bfd-978a-1e93194134ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:10.300935 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db47466f-e822-4bfd-978a-1e93194134ba None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:10.301218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db47466f-e822-4bfd-978a-1e93194134ba None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:10.301491 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db47466f-e822-4bfd-978a-1e93194134ba None admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:10.302321 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db47466f-e822-4bfd-978a-1e93194134ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:10.302548 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db47466f-e822-4bfd-978a-1e93194134ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:10.304298 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db47466f-e822-4bfd-978a-1e93194134ba None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:10.309971 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af8e82-0f57-42aa-9dd9-777d282a5fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:10.310572 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af8e82-0f57-42aa-9dd9-777d282a5fd1 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:10.310848 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af8e82-0f57-42aa-9dd9-777d282a5fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:10.311115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af8e82-0f57-42aa-9dd9-777d282a5fd1 None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:10.311760 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01af8e82-0f57-42aa-9dd9-777d282a5fd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:10.312096 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01af8e82-0f57-42aa-9dd9-777d282a5fd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:10.313445 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01af8e82-0f57-42aa-9dd9-777d282a5fd1 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:10.318112 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 306/603] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 17 23:31:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:10.318716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db47466f-e822-4bfd-978a-1e93194134ba None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:10.334356 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 298/604] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:10.334356 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af8e82-0f57-42aa-9dd9-777d282a5fd1 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:10.430897 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db47466f-e822-4bfd-978a-1e93194134ba None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:10.437907 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af8e82-0f57-42aa-9dd9-777d282a5fd1 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:10.471771 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56542374-90b2-42ea-abc3-62d8bd2b6f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:10.472470 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56542374-90b2-42ea-abc3-62d8bd2b6f61 None admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:10.472747 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56542374-90b2-42ea-abc3-62d8bd2b6f61 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:10.473018 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56542374-90b2-42ea-abc3-62d8bd2b6f61 None admin] PATH_INFO: `/v3/projects/4de86915ef4340248ba8a8f51d56594e/users/62906d50b6e247c59b69809d3959d4de/roles/b64f80e6f3ea4ad49b94acdde42ccd87` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:10.474668 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c40565-143d-499e-b14f-5f96e48c677f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:10.475391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c40565-143d-499e-b14f-5f96e48c677f None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:10.475688 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c40565-143d-499e-b14f-5f96e48c677f None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:10.475857 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:10.475989 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c40565-143d-499e-b14f-5f96e48c677f None admin] PATH_INFO: `/v3/roles/service` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:10.487950 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c40565-143d-499e-b14f-5f96e48c677f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:31:10.490586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56542374-90b2-42ea-abc3-62d8bd2b6f61 None admin] RBAC: Authorizing `identity:create_grant(project_id=4de86915ef4340248ba8a8f51d56594e, user_id=62906d50b6e247c59b69809d3959d4de, role_id=b64f80e6f3ea4ad49b94acdde42ccd87)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:10.492123 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56542374-90b2-42ea-abc3-62d8bd2b6f61 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:10.493198 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-76c40565-143d-499e-b14f-5f96e48c677f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:31:10.493198 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:10.493198 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:10.493198 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:10.493198 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:10.493198 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:10.493198 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:10.493198 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:10.493198 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:10.493198 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:10.493198 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:10.493198 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:10.493198 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:31:10.493198 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:31:10.493198 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:31:10.493198 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:31:10.494113 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:31:10.494113 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:31:10.494113 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:31:10.494113 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:31:10.494113 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:10.494113 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:10.494113 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:10.494113 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:10.494113 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:10.494113 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:10.494113 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:10.494113 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:10.494113 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:10.494113 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:10.494113 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:10.494113 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:10.495014 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:10.495014 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:10.495014 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:10.495014 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:10.495014 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:31:10.495014 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:31:10.495014 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:31:10.495014 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:31:10.495014 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:31:10.495014 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:31:10.495014 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:31:10.495014 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:10.495014 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 299/605] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 17 23:31:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:10.495729 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c40565-143d-499e-b14f-5f96e48c677f None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:10.512235 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 307/606] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 17 23:31:10 2022] PUT /identity/v3/projects/4de86915ef4340248ba8a8f51d56594e/users/62906d50b6e247c59b69809d3959d4de/roles/b64f80e6f3ea4ad49b94acdde42ccd87 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:31:10.513319 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56542374-90b2-42ea-abc3-62d8bd2b6f61 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:10.588616 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:10.626382 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56542374-90b2-42ea-abc3-62d8bd2b6f61 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:10.652941 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c40565-143d-499e-b14f-5f96e48c677f None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:10.675318 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb1abd-9dcb-424e-ae87-c2b39c4cedec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:10.676347 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb1abd-9dcb-424e-ae87-c2b39c4cedec None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:10.676696 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb1abd-9dcb-424e-ae87-c2b39c4cedec None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:10.677034 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb1abd-9dcb-424e-ae87-c2b39c4cedec None admin] PATH_INFO: `/v3/projects/service` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:10.686262 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeb1abd-9dcb-424e-ae87-c2b39c4cedec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:10.688537 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeb1abd-9dcb-424e-ae87-c2b39c4cedec None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:10.705022 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5acc99-b7b5-4139-aaec-3dfb1d7bff93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:10.705671 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5acc99-b7b5-4139-aaec-3dfb1d7bff93 None admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:10.705951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5acc99-b7b5-4139-aaec-3dfb1d7bff93 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:10.706221 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5acc99-b7b5-4139-aaec-3dfb1d7bff93 None admin] PATH_INFO: `/v3/projects/4de86915ef4340248ba8a8f51d56594e/users/ec1dcbe43b0749c3bcc4af9100d771d1/roles/b64f80e6f3ea4ad49b94acdde42ccd87` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:10.706322 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-5aeb1abd-9dcb-424e-ae87-c2b39c4cedec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:10.706322 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:10.706322 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:10.706322 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:10.706322 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:10.706322 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:10.706322 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:10.706322 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:10.706322 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:10.706322 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:10.706322 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:10.706322 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:10.706322 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 23:31:10.706322 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 23:31:10.706322 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 23:31:10.707172 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 23:31:10.707172 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:10.707172 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:10.707172 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:10.707172 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:10.707172 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:10.707172 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:10.707172 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:10.707172 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:10.707172 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:10.707172 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:10.707172 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:10.707172 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:10.707172 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:10.707172 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:10.707172 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:10.707172 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:10.708080 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 23:31:10.708080 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 23:31:10.708080 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 23:31:10.708080 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 23:31:10.708080 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 23:31:10.708080 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 23:31:10.708080 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:10.708080 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:10.708596 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 308/607] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:31:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:10.709305 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb1abd-9dcb-424e-ae87-c2b39c4cedec None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:10.724930 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5acc99-b7b5-4139-aaec-3dfb1d7bff93 None admin] RBAC: Authorizing `identity:create_grant(project_id=4de86915ef4340248ba8a8f51d56594e, user_id=ec1dcbe43b0749c3bcc4af9100d771d1, role_id=b64f80e6f3ea4ad49b94acdde42ccd87)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:10.726387 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5acc99-b7b5-4139-aaec-3dfb1d7bff93 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:10.748339 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 300/608] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 17 23:31:10 2022] PUT /identity/v3/projects/4de86915ef4340248ba8a8f51d56594e/users/ec1dcbe43b0749c3bcc4af9100d771d1/roles/b64f80e6f3ea4ad49b94acdde42ccd87 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:31:10.749372 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5acc99-b7b5-4139-aaec-3dfb1d7bff93 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:10.767595 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:10.846357 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb1abd-9dcb-424e-ae87-c2b39c4cedec None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:10.894622 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:10.899201 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd784e0-7681-4c64-abc7-d0f6f914e531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:10.899727 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5acc99-b7b5-4139-aaec-3dfb1d7bff93 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:10.899840 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd784e0-7681-4c64-abc7-d0f6f914e531 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:10.900139 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd784e0-7681-4c64-abc7-d0f6f914e531 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:10.900416 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd784e0-7681-4c64-abc7-d0f6f914e531 None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:10.901101 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd784e0-7681-4c64-abc7-d0f6f914e531 None admin] RBAC: Adding query filter params (name=service) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:10.902139 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd784e0-7681-4c64-abc7-d0f6f914e531 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:10.904790 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd784e0-7681-4c64-abc7-d0f6f914e531 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:10.918014 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 309/609] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 17 23:31:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:10.919748 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d883856-0006-4733-bf1c-a1b6401a2911 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:10.920062 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d883856-0006-4733-bf1c-a1b6401a2911 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:10.920883 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d883856-0006-4733-bf1c-a1b6401a2911 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:10.924057 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 310/610] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:10.924981 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d883856-0006-4733-bf1c-a1b6401a2911 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:10.953132 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b800a-9529-4f68-8027-e87a72a99913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:10.953370 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b800a-9529-4f68-8027-e87a72a99913 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:10.953609 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b800a-9529-4f68-8027-e87a72a99913 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:10.953769 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b800a-9529-4f68-8027-e87a72a99913 None admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:10.954476 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b800a-9529-4f68-8027-e87a72a99913 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:10.955241 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b800a-9529-4f68-8027-e87a72a99913 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:10.957618 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b800a-9529-4f68-8027-e87a72a99913 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:10.974310 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 301/611] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 17 23:31:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:10.975624 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a7f4d2-d14e-4b23-b4e7-59e86bdfcffd None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:10.975914 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a7f4d2-d14e-4b23-b4e7-59e86bdfcffd None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:10.976212 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a7f4d2-d14e-4b23-b4e7-59e86bdfcffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:10.998237 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-45a7f4d2-d14e-4b23-b4e7-59e86bdfcffd None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:10.999763 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-45a7f4d2-d14e-4b23-b4e7-59e86bdfcffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:11.055894 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d883856-0006-4733-bf1c-a1b6401a2911 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:11.065755 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:11.083945 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 302/612] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:31:11.085437 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a7f4d2-d14e-4b23-b4e7-59e86bdfcffd None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:11.101249 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0395961c-ec38-443e-80ac-0529039bd94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:11.102102 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395961c-ec38-443e-80ac-0529039bd94b None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:11.102692 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395961c-ec38-443e-80ac-0529039bd94b None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:11.103113 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395961c-ec38-443e-80ac-0529039bd94b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:11.117384 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0395961c-ec38-443e-80ac-0529039bd94b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:11.119890 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0395961c-ec38-443e-80ac-0529039bd94b None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:11.150037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-0395961c-ec38-443e-80ac-0529039bd94b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:11.150037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:11.150037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:11.150037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:11.150037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:11.150037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:11.150037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:11.150037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:11.150037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:11.150037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:11.150037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:11.150037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:11.150037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:11.150037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:11.150037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:11.150037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:11.151082 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:11.151082 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:11.151082 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:11.151082 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:11.151082 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:11.151082 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:11.151082 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:11.151082 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:11.151082 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:11.151082 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:11.151082 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:11.151082 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:11.151082 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:11.151082 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:11.151082 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:11.151082 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:11.151082 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:11.152606 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:11.152606 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:11.152606 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:11.153814 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 311/613] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:11.158953 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0395961c-ec38-443e-80ac-0529039bd94b None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:11.236362 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:11.255633 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a7f4d2-d14e-4b23-b4e7-59e86bdfcffd None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:11.295068 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb99f53f-108c-4d68-8551-8605836dcdd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:11.295783 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99f53f-108c-4d68-8551-8605836dcdd6 None admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:11.296100 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99f53f-108c-4d68-8551-8605836dcdd6 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:11.296429 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99f53f-108c-4d68-8551-8605836dcdd6 None admin] PATH_INFO: `/v3/projects/4de86915ef4340248ba8a8f51d56594e/users/16e2758f1cd144a98b98d9be0fa46ae2/roles/b64f80e6f3ea4ad49b94acdde42ccd87` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:11.312702 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb99f53f-108c-4d68-8551-8605836dcdd6 None admin] RBAC: Authorizing `identity:create_grant(project_id=4de86915ef4340248ba8a8f51d56594e, user_id=16e2758f1cd144a98b98d9be0fa46ae2, role_id=b64f80e6f3ea4ad49b94acdde42ccd87)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:11.314169 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb99f53f-108c-4d68-8551-8605836dcdd6 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:11.331378 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 303/614] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 17 23:31:11 2022] PUT /identity/v3/projects/4de86915ef4340248ba8a8f51d56594e/users/16e2758f1cd144a98b98d9be0fa46ae2/roles/b64f80e6f3ea4ad49b94acdde42ccd87 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:31:11.386290 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb99f53f-108c-4d68-8551-8605836dcdd6 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:11.403695 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0395961c-ec38-443e-80ac-0529039bd94b None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:11.478837 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d522b7-f45e-4141-8ac6-65f9c6fb6719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:11.480128 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d522b7-f45e-4141-8ac6-65f9c6fb6719 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:11.480476 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d522b7-f45e-4141-8ac6-65f9c6fb6719 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:11.480757 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d522b7-f45e-4141-8ac6-65f9c6fb6719 None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:11.481422 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d522b7-f45e-4141-8ac6-65f9c6fb6719 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:11.481730 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d522b7-f45e-4141-8ac6-65f9c6fb6719 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:11.483134 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d522b7-f45e-4141-8ac6-65f9c6fb6719 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:11.514490 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 312/615] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:31:11.521075 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d522b7-f45e-4141-8ac6-65f9c6fb6719 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:11.550263 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb99f53f-108c-4d68-8551-8605836dcdd6 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:11.551633 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:11.596223 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a83bc-a7a1-4333-aa05-b8e85b39fee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:11.596858 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a83bc-a7a1-4333-aa05-b8e85b39fee1 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:11.597143 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a83bc-a7a1-4333-aa05-b8e85b39fee1 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:11.597456 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a83bc-a7a1-4333-aa05-b8e85b39fee1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:11.607295 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:11.611864 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a83bc-a7a1-4333-aa05-b8e85b39fee1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:31:11.618642 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-748a83bc-a7a1-4333-aa05-b8e85b39fee1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:31:11.618642 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:11.618642 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:11.618642 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:11.618642 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:11.618642 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:11.618642 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:11.618642 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:11.618642 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:11.618642 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:11.618642 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:11.618642 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:11.618642 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:31:11.618642 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:31:11.618642 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:31:11.618642 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:31:11.619591 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:31:11.619591 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:31:11.619591 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:31:11.619591 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:31:11.619591 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:11.619591 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:11.619591 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:11.619591 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:11.619591 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:11.619591 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:11.619591 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:11.619591 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:11.619591 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:11.619591 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:11.619591 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:11.619591 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:11.620550 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:11.620550 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:11.620550 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:11.620550 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:11.620550 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:31:11.620550 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:31:11.620550 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:31:11.620550 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:31:11.620550 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:31:11.620550 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:31:11.620550 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:31:11.620550 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:11.620550 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 304/616] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 17 23:31:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:31:11.626186 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a83bc-a7a1-4333-aa05-b8e85b39fee1 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:11.648948 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d522b7-f45e-4141-8ac6-65f9c6fb6719 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:11.693801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1b0f9-7683-42e1-af4f-6453767b2550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:11.694806 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1b0f9-7683-42e1-af4f-6453767b2550 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:11.695290 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1b0f9-7683-42e1-af4f-6453767b2550 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:11.695764 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1b0f9-7683-42e1-af4f-6453767b2550 None admin] PATH_INFO: `/v3/users/swift` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:11.709267 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f1b0f9-7683-42e1-af4f-6453767b2550 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:11.711310 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f1b0f9-7683-42e1-af4f-6453767b2550 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:11.731930 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-41f1b0f9-7683-42e1-af4f-6453767b2550 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 17 23:31:11.731930 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:11.731930 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:11.731930 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:11.731930 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:11.731930 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:11.731930 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:11.731930 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:11.731930 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:11.731930 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:11.731930 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:11.731930 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:11.731930 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 23:31:11.731930 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 23:31:11.731930 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 23:31:11.731930 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 23:31:11.732950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:11.732950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:11.732950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 23:31:11.732950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:11.732950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 23:31:11.732950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:11.732950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:11.732950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:11.732950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:11.732950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:11.732950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:11.732950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:11.732950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:11.732950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:11.732950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:11.732950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:11.732950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:11.732950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:11.733938 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:11.733938 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:11.733938 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 23:31:11.733938 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 23:31:11.733938 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 23:31:11.733938 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 23:31:11.733938 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 23:31:11.733938 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 23:31:11.733938 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 17 23:31:11.733938 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:11.736509 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 313/617] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 17 23:31:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:11.741589 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1b0f9-7683-42e1-af4f-6453767b2550 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:11.847847 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a83bc-a7a1-4333-aa05-b8e85b39fee1 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:11.848756 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:11.875897 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1b0f9-7683-42e1-af4f-6453767b2550 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:11.925586 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e12c68-df39-44bd-87cd-0184d44934f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:11.927426 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e12c68-df39-44bd-87cd-0184d44934f5 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:11.928421 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e12c68-df39-44bd-87cd-0184d44934f5 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:11.929313 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e12c68-df39-44bd-87cd-0184d44934f5 None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:11.930501 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e12c68-df39-44bd-87cd-0184d44934f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:11.931573 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e12c68-df39-44bd-87cd-0184d44934f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:11.932929 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef38286-5199-45b2-baef-230b7a9ba2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:11.933880 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef38286-5199-45b2-baef-230b7a9ba2b9 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:11.935634 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef38286-5199-45b2-baef-230b7a9ba2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:11.936147 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef38286-5199-45b2-baef-230b7a9ba2b9 None admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:11.936575 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e12c68-df39-44bd-87cd-0184d44934f5 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:11.937510 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef38286-5199-45b2-baef-230b7a9ba2b9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:11.939485 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef38286-5199-45b2-baef-230b7a9ba2b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:11.942821 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef38286-5199-45b2-baef-230b7a9ba2b9 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:11.956505 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 305/618] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 17 23:31:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:31:11.964959 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e12c68-df39-44bd-87cd-0184d44934f5 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:11.970011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 314/619] 10.222.0.26 () {56 vars in 1135 bytes} [Wed Aug 17 23:31:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:11.975154 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef38286-5199-45b2-baef-230b7a9ba2b9 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:12.031531 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:12.082016 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e12c68-df39-44bd-87cd-0184d44934f5 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:12.127340 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef38286-5199-45b2-baef-230b7a9ba2b9 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:12.155586 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c8b7d-4f05-442a-89d8-e66445db74b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:12.156249 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c8b7d-4f05-442a-89d8-e66445db74b6 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:12.156532 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c8b7d-4f05-442a-89d8-e66445db74b6 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:12.156804 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c8b7d-4f05-442a-89d8-e66445db74b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:12.179907 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474c8b7d-4f05-442a-89d8-e66445db74b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:12.181332 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474c8b7d-4f05-442a-89d8-e66445db74b6 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:12.194569 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d06d5-6470-494d-a57b-73d1abbc3444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:12.195213 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d06d5-6470-494d-a57b-73d1abbc3444 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:12.195494 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d06d5-6470-494d-a57b-73d1abbc3444 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:12.195772 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d06d5-6470-494d-a57b-73d1abbc3444 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:12.201653 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:12.213875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-474c8b7d-4f05-442a-89d8-e66445db74b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:12.213875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:12.213875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:12.213875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:12.213875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:12.213875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:12.213875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:12.213875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:12.213875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:12.213875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:12.213875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:12.213875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:12.213875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:12.213875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:12.213875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:12.213875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:12.214844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:12.214844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:12.214844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:12.214844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:12.214844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:12.214844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:12.214844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:12.214844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:12.214844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:12.214844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:12.214844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:12.214844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:12.214844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:12.214844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:12.214844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:12.214844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:12.214844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:12.215751 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:12.215751 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:12.215751 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:12.217518 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 306/620] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:12.217646 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95d06d5-6470-494d-a57b-73d1abbc3444 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:12.218697 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95d06d5-6470-494d-a57b-73d1abbc3444 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:12.220609 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c8b7d-4f05-442a-89d8-e66445db74b6 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:12.245727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-b95d06d5-6470-494d-a57b-73d1abbc3444 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:12.245727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:12.245727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:12.245727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:12.245727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:12.245727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:12.245727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:12.245727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:12.245727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:12.245727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:12.245727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:12.245727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:12.245727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:12.245727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:12.245727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:12.245727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:12.246717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:12.246717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:12.246717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:12.246717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:12.246717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:12.246717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:12.246717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:12.246717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:12.246717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:12.246717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:12.246717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:12.246717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:12.246717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:12.246717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:12.246717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:12.246717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:12.246717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:12.247631 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:12.247631 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:12.247631 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:12.248730 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 315/621] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:31:12.256944 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d06d5-6470-494d-a57b-73d1abbc3444 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:12.367267 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c8b7d-4f05-442a-89d8-e66445db74b6 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:12.382092 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:12.399756 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d06d5-6470-494d-a57b-73d1abbc3444 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:12.406597 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5c3afd-3bbf-45a6-9b16-fc13fb6b6207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:12.407468 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c3afd-3bbf-45a6-9b16-fc13fb6b6207 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:12.407996 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c3afd-3bbf-45a6-9b16-fc13fb6b6207 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:12.408479 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c3afd-3bbf-45a6-9b16-fc13fb6b6207 None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:12.409340 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5c3afd-3bbf-45a6-9b16-fc13fb6b6207 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:12.409861 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5c3afd-3bbf-45a6-9b16-fc13fb6b6207 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:12.411440 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5c3afd-3bbf-45a6-9b16-fc13fb6b6207 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:12.424894 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 307/622] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:12.430553 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5c3afd-3bbf-45a6-9b16-fc13fb6b6207 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:12.435907 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d0fee-d380-41bb-a3bd-eddcf76a0a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:12.436559 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d0fee-d380-41bb-a3bd-eddcf76a0a93 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:12.436957 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d0fee-d380-41bb-a3bd-eddcf76a0a93 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:12.437236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d0fee-d380-41bb-a3bd-eddcf76a0a93 None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:12.437880 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7d0fee-d380-41bb-a3bd-eddcf76a0a93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:12.438182 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7d0fee-d380-41bb-a3bd-eddcf76a0a93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:12.439540 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7d0fee-d380-41bb-a3bd-eddcf76a0a93 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:12.452261 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 316/623] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:12.461027 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d0fee-d380-41bb-a3bd-eddcf76a0a93 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:12.562632 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5c3afd-3bbf-45a6-9b16-fc13fb6b6207 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:12.564288 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:12.599821 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ab457-c28f-4375-a8c9-912f59ff54b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:12.600907 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ab457-c28f-4375-a8c9-912f59ff54b0 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:12.601548 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ab457-c28f-4375-a8c9-912f59ff54b0 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:12.602344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ab457-c28f-4375-a8c9-912f59ff54b0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:12.611075 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9ab457-c28f-4375-a8c9-912f59ff54b0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:12.612955 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9ab457-c28f-4375-a8c9-912f59ff54b0 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:12.626765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-cd9ab457-c28f-4375-a8c9-912f59ff54b0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 17 23:31:12.626765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:12.626765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:12.626765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:12.626765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:12.626765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:12.626765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:12.626765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:12.626765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:12.626765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:12.626765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:12.626765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:12.626765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 23:31:12.626765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 23:31:12.626765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 23:31:12.626765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 23:31:12.627716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:12.627716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:12.627716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 23:31:12.627716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:12.627716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 23:31:12.627716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:12.627716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:12.627716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:12.627716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:12.627716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:12.627716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:12.627716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:12.627716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:12.627716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:12.627716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:12.627716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:12.627716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:12.627716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:12.628693 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:12.628693 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:12.628693 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 23:31:12.628693 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 23:31:12.628693 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 23:31:12.628693 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 23:31:12.628693 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 23:31:12.628693 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 23:31:12.628693 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 17 23:31:12.628693 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:12.630506 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 308/624] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:31:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:31:12.636962 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ab457-c28f-4375-a8c9-912f59ff54b0 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:12.675034 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:12.733905 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d0fee-d380-41bb-a3bd-eddcf76a0a93 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:12.761107 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ab457-c28f-4375-a8c9-912f59ff54b0 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:12.772616 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ada34d-e570-45b4-8983-e0fd1fd21fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:12.773251 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ada34d-e570-45b4-8983-e0fd1fd21fda None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:12.773531 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ada34d-e570-45b4-8983-e0fd1fd21fda None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:12.773807 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ada34d-e570-45b4-8983-e0fd1fd21fda None admin] PATH_INFO: `/v3/projects/service` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:12.781553 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ada34d-e570-45b4-8983-e0fd1fd21fda None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:12.783016 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ada34d-e570-45b4-8983-e0fd1fd21fda None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:12.795058 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-d5ada34d-e570-45b4-8983-e0fd1fd21fda None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:12.795058 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:12.795058 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:12.795058 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:12.795058 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:12.795058 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:12.795058 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:12.795058 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:12.795058 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:12.795058 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:12.795058 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:12.795058 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:12.795058 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 23:31:12.795058 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 23:31:12.795058 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 23:31:12.796000 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 23:31:12.796000 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:12.796000 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:12.796000 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:12.796000 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:12.796000 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:12.796000 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:12.796000 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:12.796000 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:12.796000 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:12.796000 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:12.796000 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:12.796000 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:12.796000 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:12.796000 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:12.796000 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:12.796000 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:12.796907 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 23:31:12.796907 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 23:31:12.796907 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 23:31:12.796907 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 23:31:12.796907 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 23:31:12.796907 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 23:31:12.796907 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:12.796907 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:12.796907 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 317/625] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:31:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:12.797855 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de2c61-a26b-48b6-b970-f4d2e6eab9c9 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:12.798018 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de2c61-a26b-48b6-b970-f4d2e6eab9c9 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:12.798298 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de2c61-a26b-48b6-b970-f4d2e6eab9c9 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:12.799795 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 318/626] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:12.807925 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de2c61-a26b-48b6-b970-f4d2e6eab9c9 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:12.811397 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd01c1c-4a05-4cd4-8cb7-e4ca2c58365f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:12.812163 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd01c1c-4a05-4cd4-8cb7-e4ca2c58365f None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:12.812548 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd01c1c-4a05-4cd4-8cb7-e4ca2c58365f None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:12.812941 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd01c1c-4a05-4cd4-8cb7-e4ca2c58365f None admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:12.813780 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd01c1c-4a05-4cd4-8cb7-e4ca2c58365f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:12.814192 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd01c1c-4a05-4cd4-8cb7-e4ca2c58365f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:12.815738 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd01c1c-4a05-4cd4-8cb7-e4ca2c58365f None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:12.834389 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 309/627] 10.222.0.26 () {56 vars in 1137 bytes} [Wed Aug 17 23:31:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:12.835955 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee202d4e-3ab1-4101-b606-d0489cf8822f None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:12.836443 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee202d4e-3ab1-4101-b606-d0489cf8822f None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:12.836821 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee202d4e-3ab1-4101-b606-d0489cf8822f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:12.857228 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-ee202d4e-3ab1-4101-b606-d0489cf8822f None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:12.858590 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-ee202d4e-3ab1-4101-b606-d0489cf8822f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:12.932164 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 310/628] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:31:12.937585 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de2c61-a26b-48b6-b970-f4d2e6eab9c9 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:12.937816 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee202d4e-3ab1-4101-b606-d0489cf8822f None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:12.959349 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:12.990426 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:13.013353 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2a5769-a2b1-4d16-b037-95437d95e13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:13.014213 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a5769-a2b1-4d16-b037-95437d95e13b None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:13.016269 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a5769-a2b1-4d16-b037-95437d95e13b None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:13.018938 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a5769-a2b1-4d16-b037-95437d95e13b None admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:13.019822 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2a5769-a2b1-4d16-b037-95437d95e13b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:13.020350 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2a5769-a2b1-4d16-b037-95437d95e13b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:13.022005 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2a5769-a2b1-4d16-b037-95437d95e13b None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:13.040139 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 319/629] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 17 23:31:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:13.042430 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac603c-4e45-4b3a-a492-48b491a2da33 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:13.042895 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac603c-4e45-4b3a-a492-48b491a2da33 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:13.043614 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac603c-4e45-4b3a-a492-48b491a2da33 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:13.045970 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 320/630] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:13.046820 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac603c-4e45-4b3a-a492-48b491a2da33 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:13.100888 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee202d4e-3ab1-4101-b606-d0489cf8822f None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:13.144696 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a546e-9592-4d32-b5b6-49fb8ecb910e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:13.145714 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a546e-9592-4d32-b5b6-49fb8ecb910e None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:13.146342 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a546e-9592-4d32-b5b6-49fb8ecb910e None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:13.146797 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a546e-9592-4d32-b5b6-49fb8ecb910e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:13.155092 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac603c-4e45-4b3a-a492-48b491a2da33 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:13.192044 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f184bc-7f8e-4202-a888-c973cfdeae61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:13.192816 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f184bc-7f8e-4202-a888-c973cfdeae61 None admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:13.193183 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f184bc-7f8e-4202-a888-c973cfdeae61 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:13.194243 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f184bc-7f8e-4202-a888-c973cfdeae61 None admin] PATH_INFO: `/v3/projects/4de86915ef4340248ba8a8f51d56594e/users/ee422ea90557444fabdc9c292865b9b8/roles/b64f80e6f3ea4ad49b94acdde42ccd87` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:13.209442 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f184bc-7f8e-4202-a888-c973cfdeae61 None admin] RBAC: Authorizing `identity:create_grant(project_id=4de86915ef4340248ba8a8f51d56594e, user_id=ee422ea90557444fabdc9c292865b9b8, role_id=b64f80e6f3ea4ad49b94acdde42ccd87)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:13.210886 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f184bc-7f8e-4202-a888-c973cfdeae61 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:13.224701 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 321/631] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 17 23:31:13 2022] PUT /identity/v3/projects/4de86915ef4340248ba8a8f51d56594e/users/ee422ea90557444fabdc9c292865b9b8/roles/b64f80e6f3ea4ad49b94acdde42ccd87 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:31:13.226002 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f083306-7f92-4049-b45c-1558733b1e3b None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:13.226294 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f083306-7f92-4049-b45c-1558733b1e3b None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:13.226568 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f083306-7f92-4049-b45c-1558733b1e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:13.243674 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a546e-9592-4d32-b5b6-49fb8ecb910e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:13.245893 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a546e-9592-4d32-b5b6-49fb8ecb910e None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:13.246020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-4f083306-7f92-4049-b45c-1558733b1e3b None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:13.247324 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-4f083306-7f92-4049-b45c-1558733b1e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:13.263994 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:13.277433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:13.336197 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 322/632] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:31:13.337533 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f083306-7f92-4049-b45c-1558733b1e3b None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:13.398453 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 311/633] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:31:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:31:13.406112 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a546e-9592-4d32-b5b6-49fb8ecb910e None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:13.494629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f083306-7f92-4049-b45c-1558733b1e3b None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:13.540599 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ba00a-6bb1-4dd1-a25e-6423a67a14c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:13.542324 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ba00a-6bb1-4dd1-a25e-6423a67a14c4 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:13.543085 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ba00a-6bb1-4dd1-a25e-6423a67a14c4 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:13.543563 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ba00a-6bb1-4dd1-a25e-6423a67a14c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:13.544840 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a546e-9592-4d32-b5b6-49fb8ecb910e None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:13.560909 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740ba00a-6bb1-4dd1-a25e-6423a67a14c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:31:13.567526 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-740ba00a-6bb1-4dd1-a25e-6423a67a14c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:31:13.567526 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:13.567526 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:13.567526 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:13.567526 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:13.567526 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:13.567526 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:13.567526 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:13.567526 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:13.567526 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:13.567526 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:13.567526 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:13.567526 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:31:13.567526 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:31:13.567526 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:31:13.567526 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:31:13.568492 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:31:13.568492 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:31:13.568492 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:31:13.568492 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:31:13.568492 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:13.568492 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:13.568492 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:13.568492 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:13.568492 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:13.568492 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:13.568492 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:13.568492 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:13.568492 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:13.568492 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:13.568492 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:13.568492 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:13.569998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:13.569998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:13.569998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:13.569998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:13.569998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:31:13.569998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:31:13.569998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:31:13.569998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:31:13.569998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:31:13.569998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:31:13.569998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:31:13.569998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:13.569998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 323/634] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 17 23:31:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:13.576933 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ba00a-6bb1-4dd1-a25e-6423a67a14c4 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:13.588555 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:13.589958 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:13.624252 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e697ef-4891-4080-8b6b-f5a6eaa96383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:13.624884 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e697ef-4891-4080-8b6b-f5a6eaa96383 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:13.625170 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e697ef-4891-4080-8b6b-f5a6eaa96383 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:13.625446 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e697ef-4891-4080-8b6b-f5a6eaa96383 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:13.649926 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e697ef-4891-4080-8b6b-f5a6eaa96383 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:13.651423 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e697ef-4891-4080-8b6b-f5a6eaa96383 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:13.681279 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-15e697ef-4891-4080-8b6b-f5a6eaa96383 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:13.681279 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:13.681279 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:13.681279 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:13.681279 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:13.681279 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:13.681279 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:13.681279 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:13.681279 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:13.681279 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:13.681279 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:13.681279 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:13.681279 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:13.681279 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:13.681279 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:13.681279 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:13.682291 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:13.682291 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:13.682291 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:13.682291 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:13.682291 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:13.682291 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:13.682291 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:13.682291 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:13.682291 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:13.682291 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:13.682291 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:13.682291 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:13.682291 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:13.682291 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:13.682291 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:13.682291 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:13.682291 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:13.684325 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:13.684325 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:13.684325 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:13.684325 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 312/635] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:31:13.692950 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e697ef-4891-4080-8b6b-f5a6eaa96383 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:13.694307 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ba00a-6bb1-4dd1-a25e-6423a67a14c4 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:13.732675 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b5a95-ae81-4070-b824-df14a4d39bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:13.733273 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b5a95-ae81-4070-b824-df14a4d39bdf None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:13.733546 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b5a95-ae81-4070-b824-df14a4d39bdf None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:13.733815 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b5a95-ae81-4070-b824-df14a4d39bdf None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:13.734477 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0b5a95-ae81-4070-b824-df14a4d39bdf None admin] RBAC: Adding query filter params (name=service) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:13.734773 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0b5a95-ae81-4070-b824-df14a4d39bdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:13.736246 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0b5a95-ae81-4070-b824-df14a4d39bdf None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:13.748244 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 324/636] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 17 23:31:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:13.748244 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b5a95-ae81-4070-b824-df14a4d39bdf None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:13.800103 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e697ef-4891-4080-8b6b-f5a6eaa96383 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:13.818235 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:13.835334 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ff159-0b4f-4d04-a484-db3b2a4fa2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:13.836069 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ff159-0b4f-4d04-a484-db3b2a4fa2d3 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:13.836408 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ff159-0b4f-4d04-a484-db3b2a4fa2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:13.836743 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ff159-0b4f-4d04-a484-db3b2a4fa2d3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:13.847855 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823ff159-0b4f-4d04-a484-db3b2a4fa2d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:31:13.851947 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b5a95-ae81-4070-b824-df14a4d39bdf None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:13.853207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-823ff159-0b4f-4d04-a484-db3b2a4fa2d3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:31:13.853207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:13.853207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:13.853207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:13.853207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:13.853207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:13.853207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:13.853207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:13.853207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:13.853207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:13.853207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:13.853207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:13.853207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:31:13.853207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:31:13.853207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:31:13.853207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:31:13.854187 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:31:13.854187 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:31:13.854187 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:31:13.854187 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:31:13.854187 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:13.854187 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:13.854187 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:13.854187 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:13.854187 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:13.854187 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:13.854187 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:13.854187 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:13.854187 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:13.854187 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:13.854187 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:13.854187 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:13.856452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:13.856452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:13.856452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:13.856452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:13.856452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:31:13.856452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:31:13.856452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:31:13.856452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:31:13.856452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:31:13.856452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:31:13.856452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:31:13.856452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:13.856452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 313/637] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 17 23:31:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:13.856452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd126687-7f8a-4c54-b150-97b4175c74ed None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:13.857213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd126687-7f8a-4c54-b150-97b4175c74ed None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:13.857213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd126687-7f8a-4c54-b150-97b4175c74ed None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:13.858126 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 314/638] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:13.858655 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd126687-7f8a-4c54-b150-97b4175c74ed None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:13.891657 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:13.900981 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca761004-efb3-4f7f-80e7-74e316ca29c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:13.901625 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca761004-efb3-4f7f-80e7-74e316ca29c3 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:13.902341 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca761004-efb3-4f7f-80e7-74e316ca29c3 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:13.902616 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca761004-efb3-4f7f-80e7-74e316ca29c3 None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:13.903531 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca761004-efb3-4f7f-80e7-74e316ca29c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:13.904220 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca761004-efb3-4f7f-80e7-74e316ca29c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:13.905937 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca761004-efb3-4f7f-80e7-74e316ca29c3 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:13.918927 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 325/639] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:13.919734 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca761004-efb3-4f7f-80e7-74e316ca29c3 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:13.968718 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd126687-7f8a-4c54-b150-97b4175c74ed None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:14.007031 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1623f3f2-2bfc-49fd-ad0d-cb3df0c11135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:14.007586 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623f3f2-2bfc-49fd-ad0d-cb3df0c11135 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:14.007866 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623f3f2-2bfc-49fd-ad0d-cb3df0c11135 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:14.008186 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623f3f2-2bfc-49fd-ad0d-cb3df0c11135 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:14.019281 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1623f3f2-2bfc-49fd-ad0d-cb3df0c11135 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:14.020706 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1623f3f2-2bfc-49fd-ad0d-cb3df0c11135 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:14.024809 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca761004-efb3-4f7f-80e7-74e316ca29c3 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:14.036076 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-1623f3f2-2bfc-49fd-ad0d-cb3df0c11135 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:14.036076 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:14.036076 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:14.036076 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:14.036076 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:14.036076 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:14.036076 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:14.036076 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:14.036076 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:14.036076 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:14.036076 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:14.036076 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:14.036076 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:14.036076 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:14.036076 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:14.036076 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:14.037050 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:14.037050 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:14.037050 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:14.037050 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:14.037050 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:14.037050 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:14.037050 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:14.037050 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:14.037050 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:14.037050 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:14.037050 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:14.037050 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:14.037050 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:14.037050 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:14.037050 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:14.037050 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:14.037050 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:14.037936 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:14.037936 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:14.037936 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:14.037936 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 315/640] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:14.038871 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31e887-3e5c-4b83-9476-0106e0750282 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:14.038871 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31e887-3e5c-4b83-9476-0106e0750282 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:14.039043 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31e887-3e5c-4b83-9476-0106e0750282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:14.057487 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-8f31e887-3e5c-4b83-9476-0106e0750282 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:14.058653 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-8f31e887-3e5c-4b83-9476-0106e0750282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:14.060765 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:14.067230 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f81883-4390-44a6-b53e-0df074cbe3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:14.067839 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f81883-4390-44a6-b53e-0df074cbe3b2 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:14.068146 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f81883-4390-44a6-b53e-0df074cbe3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:14.068427 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f81883-4390-44a6-b53e-0df074cbe3b2 None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:14.069095 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f81883-4390-44a6-b53e-0df074cbe3b2 None admin] RBAC: Adding query filter params (name=service) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:14.069401 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f81883-4390-44a6-b53e-0df074cbe3b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:14.070766 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f81883-4390-44a6-b53e-0df074cbe3b2 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:14.079689 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 326/641] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 17 23:31:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:14.084758 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f81883-4390-44a6-b53e-0df074cbe3b2 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:14.129189 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 316/642] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:31:14.130293 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f31e887-3e5c-4b83-9476-0106e0750282 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:14.146362 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:14.214332 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f81883-4390-44a6-b53e-0df074cbe3b2 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:14.240865 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f31e887-3e5c-4b83-9476-0106e0750282 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:14.257662 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95edee09-f633-4770-90a4-b6a2f5cffcd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:14.258226 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95edee09-f633-4770-90a4-b6a2f5cffcd7 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:14.258561 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95edee09-f633-4770-90a4-b6a2f5cffcd7 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:14.258863 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95edee09-f633-4770-90a4-b6a2f5cffcd7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:14.266493 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95edee09-f633-4770-90a4-b6a2f5cffcd7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:14.268110 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95edee09-f633-4770-90a4-b6a2f5cffcd7 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:14.279914 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec573ea7-907e-4c4d-9e85-0801dd52929b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:14.280204 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-95edee09-f633-4770-90a4-b6a2f5cffcd7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:14.280204 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:14.280204 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:14.280204 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:14.280204 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:14.280204 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:14.280204 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:14.280204 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:14.280204 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:14.280204 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:14.280204 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:14.280204 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:14.280204 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 23:31:14.280204 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 23:31:14.280204 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 23:31:14.281303 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 23:31:14.281303 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:14.281303 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:14.281303 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:14.281303 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:14.281303 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:14.281303 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:14.281303 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:14.281303 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:14.281303 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:14.281303 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:14.281303 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:14.281303 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:14.281303 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:14.281303 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:14.281303 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:14.281303 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:14.282425 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 23:31:14.282425 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 23:31:14.282425 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 23:31:14.282425 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 23:31:14.282425 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 23:31:14.282425 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 23:31:14.282425 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:14.282425 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:14.282425 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 327/643] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:31:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:14.282425 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95edee09-f633-4770-90a4-b6a2f5cffcd7 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:14.283641 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec573ea7-907e-4c4d-9e85-0801dd52929b None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:14.284104 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec573ea7-907e-4c4d-9e85-0801dd52929b None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:14.284504 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec573ea7-907e-4c4d-9e85-0801dd52929b None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:14.285395 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec573ea7-907e-4c4d-9e85-0801dd52929b None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:14.285947 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec573ea7-907e-4c4d-9e85-0801dd52929b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:14.287548 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec573ea7-907e-4c4d-9e85-0801dd52929b None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:14.300652 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 317/644] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:14.301677 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec573ea7-907e-4c4d-9e85-0801dd52929b None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:14.345851 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:14.384601 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:14.412059 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95edee09-f633-4770-90a4-b6a2f5cffcd7 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:14.412367 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec573ea7-907e-4c4d-9e85-0801dd52929b None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:14.447539 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd626cb-d974-48af-805d-c7af2be9e860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:14.448862 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd626cb-d974-48af-805d-c7af2be9e860 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:14.449234 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd626cb-d974-48af-805d-c7af2be9e860 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:14.449595 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd626cb-d974-48af-805d-c7af2be9e860 None admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:14.450381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd626cb-d974-48af-805d-c7af2be9e860 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:14.450798 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd626cb-d974-48af-805d-c7af2be9e860 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:14.452633 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd626cb-d974-48af-805d-c7af2be9e860 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:14.458826 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09dfe23-8bde-4047-bca5-3fd92730d3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:14.459457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09dfe23-8bde-4047-bca5-3fd92730d3ed None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:14.459738 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09dfe23-8bde-4047-bca5-3fd92730d3ed None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:14.460038 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09dfe23-8bde-4047-bca5-3fd92730d3ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:14.466241 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 318/645] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 17 23:31:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:14.471593 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd626cb-d974-48af-805d-c7af2be9e860 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:14.472491 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09dfe23-8bde-4047-bca5-3fd92730d3ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:14.473961 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09dfe23-8bde-4047-bca5-3fd92730d3ed None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:14.489276 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-d09dfe23-8bde-4047-bca5-3fd92730d3ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:14.489276 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:14.489276 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:14.489276 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:14.489276 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:14.489276 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:14.489276 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:14.489276 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:14.489276 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:14.489276 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:14.489276 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:14.489276 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:14.489276 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:14.489276 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:14.489276 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:14.489276 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:14.490365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:14.490365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:14.490365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:14.490365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:14.490365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:14.490365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:14.490365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:14.490365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:14.490365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:14.490365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:14.490365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:14.490365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:14.490365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:14.490365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:14.490365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:14.490365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:14.490365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:14.497130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:14.497130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:14.497130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:14.497130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 328/646] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:14.497130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09dfe23-8bde-4047-bca5-3fd92730d3ed None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:14.585984 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd626cb-d974-48af-805d-c7af2be9e860 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:14.609825 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09dfe23-8bde-4047-bca5-3fd92730d3ed None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:14.612109 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:14.621592 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a357a-7047-4b4d-88b5-d3dc8eba9970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:14.622328 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a357a-7047-4b4d-88b5-d3dc8eba9970 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:14.622699 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a357a-7047-4b4d-88b5-d3dc8eba9970 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:14.623066 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a357a-7047-4b4d-88b5-d3dc8eba9970 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:14.625064 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:14.631543 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a357a-7047-4b4d-88b5-d3dc8eba9970 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:14.633133 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a357a-7047-4b4d-88b5-d3dc8eba9970 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:14.644520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-115a357a-7047-4b4d-88b5-d3dc8eba9970 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 17 23:31:14.644520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:14.644520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:14.644520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:14.644520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:14.644520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:14.644520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:14.644520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:14.644520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:14.644520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:14.644520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:14.644520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:14.644520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 23:31:14.644520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 23:31:14.644520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 23:31:14.644520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 23:31:14.645690 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:14.645690 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:14.645690 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 23:31:14.645690 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:14.645690 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 23:31:14.645690 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:14.645690 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:14.645690 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:14.645690 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:14.645690 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:14.645690 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:14.645690 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:14.645690 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:14.645690 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:14.645690 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:14.645690 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:14.645690 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:14.645690 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:14.646917 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:14.646917 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:14.646917 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 23:31:14.646917 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 23:31:14.646917 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 23:31:14.646917 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 23:31:14.646917 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 23:31:14.646917 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 23:31:14.646917 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 17 23:31:14.646917 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:14.648885 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de28407-fd1f-4b3a-bb9c-b9fd3006f1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:14.649612 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de28407-fd1f-4b3a-bb9c-b9fd3006f1d0 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:14.649990 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de28407-fd1f-4b3a-bb9c-b9fd3006f1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:14.650539 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de28407-fd1f-4b3a-bb9c-b9fd3006f1d0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:14.650654 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 319/647] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:31:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:14.651462 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a357a-7047-4b4d-88b5-d3dc8eba9970 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:14.661459 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de28407-fd1f-4b3a-bb9c-b9fd3006f1d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:31:14.667262 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-9de28407-fd1f-4b3a-bb9c-b9fd3006f1d0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:31:14.667262 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:14.667262 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:14.667262 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:14.667262 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:14.667262 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:14.667262 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:14.667262 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:14.667262 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:14.667262 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:14.667262 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:14.667262 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:14.667262 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:31:14.667262 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:31:14.667262 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:31:14.667262 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:31:14.668427 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:31:14.668427 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:31:14.668427 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:31:14.668427 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:31:14.668427 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:14.668427 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:14.668427 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:14.668427 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:14.668427 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:14.668427 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:14.668427 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:14.668427 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:14.668427 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:14.668427 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:14.668427 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:14.668427 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:14.669455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:14.669455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:14.669455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:14.669455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:14.669455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:31:14.669455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:31:14.669455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:31:14.669455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:31:14.669455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:31:14.669455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:31:14.669455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:31:14.669455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:14.675055 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 329/648] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 17 23:31:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:14.675757 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de28407-fd1f-4b3a-bb9c-b9fd3006f1d0 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:14.759416 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a357a-7047-4b4d-88b5-d3dc8eba9970 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:14.781003 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de28407-fd1f-4b3a-bb9c-b9fd3006f1d0 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:14.796037 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74d849-a861-4c0f-a7c7-dc9d17ba5c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:14.796714 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74d849-a861-4c0f-a7c7-dc9d17ba5c1d None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:14.797061 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74d849-a861-4c0f-a7c7-dc9d17ba5c1d None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:14.797401 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74d849-a861-4c0f-a7c7-dc9d17ba5c1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:14.798194 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd74d849-a861-4c0f-a7c7-dc9d17ba5c1d None admin] RBAC: Adding query filter params (role.id=b64f80e6f3ea4ad49b94acdde42ccd87, scope.project.id=4de86915ef4340248ba8a8f51d56594e, user.id=9c08c497a14c426f9cfe0dd8c8100a21) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:14.798571 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd74d849-a861-4c0f-a7c7-dc9d17ba5c1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:14.800091 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd74d849-a861-4c0f-a7c7-dc9d17ba5c1d None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:14.810792 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 320/649] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:31:14 2022] GET /identity/v3/role_assignments?user.id=9c08c497a14c426f9cfe0dd8c8100a21&scope.project.id=4de86915ef4340248ba8a8f51d56594e&role.id=b64f80e6f3ea4ad49b94acdde42ccd87 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:14.811279 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74d849-a861-4c0f-a7c7-dc9d17ba5c1d None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:14.831623 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c478d3-a17f-4586-997a-84a9c57522c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:14.833112 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c478d3-a17f-4586-997a-84a9c57522c5 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:14.833591 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c478d3-a17f-4586-997a-84a9c57522c5 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:14.834028 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c478d3-a17f-4586-997a-84a9c57522c5 None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:14.834869 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c478d3-a17f-4586-997a-84a9c57522c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:14.835342 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c478d3-a17f-4586-997a-84a9c57522c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:14.838003 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c478d3-a17f-4586-997a-84a9c57522c5 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:14.864583 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 330/650] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:14.865621 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c478d3-a17f-4586-997a-84a9c57522c5 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:14.886871 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:14.906943 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:14.945692 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74d849-a861-4c0f-a7c7-dc9d17ba5c1d None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:14.993193 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b1fc8-da0f-4a8c-9ba7-7a18e29044ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:14.994686 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b1fc8-da0f-4a8c-9ba7-7a18e29044ed None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:14.995514 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b1fc8-da0f-4a8c-9ba7-7a18e29044ed None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:14.995782 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b1fc8-da0f-4a8c-9ba7-7a18e29044ed None admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:14.996542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b1fc8-da0f-4a8c-9ba7-7a18e29044ed None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:14.996842 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b1fc8-da0f-4a8c-9ba7-7a18e29044ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:14.998268 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b1fc8-da0f-4a8c-9ba7-7a18e29044ed None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:15.002416 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c478d3-a17f-4586-997a-84a9c57522c5 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:15.022148 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 321/651] 10.222.0.26 () {56 vars in 1137 bytes} [Wed Aug 17 23:31:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:15.023315 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b1fc8-da0f-4a8c-9ba7-7a18e29044ed None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:15.050364 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e4f4e-e0b3-4a27-91be-909ccd50d396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:15.050994 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e4f4e-e0b3-4a27-91be-909ccd50d396 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:15.051422 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e4f4e-e0b3-4a27-91be-909ccd50d396 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:15.052401 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e4f4e-e0b3-4a27-91be-909ccd50d396 None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:15.052759 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e4f4e-e0b3-4a27-91be-909ccd50d396 None admin] RBAC: Adding query filter params (name=service) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:15.053065 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e4f4e-e0b3-4a27-91be-909ccd50d396 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:15.055088 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e4f4e-e0b3-4a27-91be-909ccd50d396 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:15.066884 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 331/652] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 17 23:31:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:15.066884 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e4f4e-e0b3-4a27-91be-909ccd50d396 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:15.148429 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b1fc8-da0f-4a8c-9ba7-7a18e29044ed None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:15.162699 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:15.168855 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:15.179303 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e4f4e-e0b3-4a27-91be-909ccd50d396 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:15.190087 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3613c9d1-2a3e-4560-8532-aeaf77e78af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:15.190713 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613c9d1-2a3e-4560-8532-aeaf77e78af9 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:15.190981 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613c9d1-2a3e-4560-8532-aeaf77e78af9 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:15.191250 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613c9d1-2a3e-4560-8532-aeaf77e78af9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:15.199981 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3613c9d1-2a3e-4560-8532-aeaf77e78af9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:15.201462 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3613c9d1-2a3e-4560-8532-aeaf77e78af9 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:15.214128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-3613c9d1-2a3e-4560-8532-aeaf77e78af9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 17 23:31:15.214128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:15.214128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:15.214128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:15.214128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:15.214128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:15.214128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:15.214128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:15.214128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:15.214128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:15.214128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:15.214128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:15.214128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 23:31:15.214128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 23:31:15.214128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 23:31:15.214128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 23:31:15.215161 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:15.215161 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:15.215161 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 23:31:15.215161 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:15.215161 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 23:31:15.215161 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:15.215161 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:15.215161 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:15.215161 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:15.215161 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:15.215161 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:15.215161 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:15.215161 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:15.215161 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:15.215161 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:15.215161 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:15.215161 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:15.215161 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:15.216114 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:15.216114 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:15.216114 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 23:31:15.216114 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 23:31:15.216114 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 23:31:15.216114 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 23:31:15.216114 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 23:31:15.216114 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 23:31:15.216114 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 17 23:31:15.216114 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:15.216114 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 322/653] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Aug 17 23:31:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:15.216114 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3613c9d1-2a3e-4560-8532-aeaf77e78af9 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:15.221111 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ad42d-fab5-41b5-9b1a-71e8b8228a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:15.222391 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ad42d-fab5-41b5-9b1a-71e8b8228a66 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:15.222866 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ad42d-fab5-41b5-9b1a-71e8b8228a66 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:15.223418 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ad42d-fab5-41b5-9b1a-71e8b8228a66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:15.315730 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ad42d-fab5-41b5-9b1a-71e8b8228a66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:15.317227 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ad42d-fab5-41b5-9b1a-71e8b8228a66 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:15.336979 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3613c9d1-2a3e-4560-8532-aeaf77e78af9 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:15.373632 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c540614-cdc6-4a1a-8b88-a9e8b2ad9ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:15.374280 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c540614-cdc6-4a1a-8b88-a9e8b2ad9ff7 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:15.374558 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c540614-cdc6-4a1a-8b88-a9e8b2ad9ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:15.374827 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c540614-cdc6-4a1a-8b88-a9e8b2ad9ff7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:15.385547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c540614-cdc6-4a1a-8b88-a9e8b2ad9ff7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:15.387006 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c540614-cdc6-4a1a-8b88-a9e8b2ad9ff7 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:15.402306 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:15.410003 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-6c540614-cdc6-4a1a-8b88-a9e8b2ad9ff7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:15.410003 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:15.410003 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:15.410003 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:15.410003 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:15.410003 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:15.410003 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:15.410003 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:15.410003 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:15.410003 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:15.410003 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:15.410003 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:15.410003 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:15.410003 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:15.410003 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:15.410003 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:15.410964 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:15.410964 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:15.410964 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:15.410964 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:15.410964 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:15.410964 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:15.410964 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:15.410964 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:15.410964 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:15.410964 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:15.410964 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:15.410964 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:15.410964 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:15.410964 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:15.410964 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:15.410964 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:15.410964 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:15.411900 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:15.411900 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:15.411900 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:15.411900 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 323/654] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:15.412179 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c540614-cdc6-4a1a-8b88-a9e8b2ad9ff7 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:15.428690 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 332/655] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:31:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:31:15.428690 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7307c-ae1b-497a-a8c4-fc66ddb50bb3 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:15.428690 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7307c-ae1b-497a-a8c4-fc66ddb50bb3 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:15.428690 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7307c-ae1b-497a-a8c4-fc66ddb50bb3 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:15.428690 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 333/656] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:15.428690 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce7307c-ae1b-497a-a8c4-fc66ddb50bb3 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:15.453045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:15.541220 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce7307c-ae1b-497a-a8c4-fc66ddb50bb3 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:15.541591 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c540614-cdc6-4a1a-8b88-a9e8b2ad9ff7 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:15.577004 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0325f215-7aa0-46c0-9648-a796667e8243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:15.577610 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325f215-7aa0-46c0-9648-a796667e8243 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:15.577890 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325f215-7aa0-46c0-9648-a796667e8243 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:15.578173 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325f215-7aa0-46c0-9648-a796667e8243 None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:15.578790 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0325f215-7aa0-46c0-9648-a796667e8243 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:15.579145 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0325f215-7aa0-46c0-9648-a796667e8243 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:15.580500 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0325f215-7aa0-46c0-9648-a796667e8243 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:15.581036 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb0965-61ba-43da-b96e-8dfeff822ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:15.581626 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb0965-61ba-43da-b96e-8dfeff822ca7 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:15.581903 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb0965-61ba-43da-b96e-8dfeff822ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:15.582174 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb0965-61ba-43da-b96e-8dfeff822ca7 None admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:15.582908 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cb0965-61ba-43da-b96e-8dfeff822ca7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:15.583211 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cb0965-61ba-43da-b96e-8dfeff822ca7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:15.584692 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cb0965-61ba-43da-b96e-8dfeff822ca7 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:15.596975 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 334/657] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:15.596975 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0325f215-7aa0-46c0-9648-a796667e8243 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:15.608047 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 324/658] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Aug 17 23:31:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:15.609158 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb89d4-9cb6-4a9a-a41f-de8c72b90ea8 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:15.609439 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb89d4-9cb6-4a9a-a41f-de8c72b90ea8 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:15.609710 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb89d4-9cb6-4a9a-a41f-de8c72b90ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:15.627588 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-9cfb89d4-9cb6-4a9a-a41f-de8c72b90ea8 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:15.628800 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-9cfb89d4-9cb6-4a9a-a41f-de8c72b90ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:15.647774 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:15.697601 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 325/659] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:31:15.698485 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb89d4-9cb6-4a9a-a41f-de8c72b90ea8 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:15.715438 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0325f215-7aa0-46c0-9648-a796667e8243 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:15.721504 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:15.758469 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6211a755-be9e-430d-93f9-576372ffa74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:15.759802 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211a755-be9e-430d-93f9-576372ffa74d None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:15.760281 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211a755-be9e-430d-93f9-576372ffa74d None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:15.761444 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211a755-be9e-430d-93f9-576372ffa74d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:15.776443 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6211a755-be9e-430d-93f9-576372ffa74d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:15.779365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6211a755-be9e-430d-93f9-576372ffa74d None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:15.797141 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-6211a755-be9e-430d-93f9-576372ffa74d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:15.797141 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:15.797141 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:15.797141 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:15.797141 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:15.797141 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:15.797141 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:15.797141 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:15.797141 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:15.797141 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:15.797141 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:15.797141 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:15.797141 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:15.797141 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:15.797141 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:15.797141 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:15.798122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:15.798122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:15.798122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:15.798122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:15.798122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:15.798122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:15.798122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:15.798122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:15.798122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:15.798122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:15.798122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:15.798122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:15.798122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:15.798122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:15.798122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:15.798122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:15.798122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:15.799007 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:15.799007 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:15.799007 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:15.799007 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 335/660] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:15.799866 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6211a755-be9e-430d-93f9-576372ffa74d None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:15.826050 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb89d4-9cb6-4a9a-a41f-de8c72b90ea8 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:15.872557 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce515742-4ad4-4997-82d8-d19af29ee1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:15.874447 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce515742-4ad4-4997-82d8-d19af29ee1ec None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:15.874893 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce515742-4ad4-4997-82d8-d19af29ee1ec None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:15.875322 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce515742-4ad4-4997-82d8-d19af29ee1ec None admin] PATH_INFO: `/v3/users/neutron` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:15.884458 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce515742-4ad4-4997-82d8-d19af29ee1ec None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:15.888513 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce515742-4ad4-4997-82d8-d19af29ee1ec None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:15.903328 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:15.906245 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-ce515742-4ad4-4997-82d8-d19af29ee1ec None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 17 23:31:15.906245 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:15.906245 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:15.906245 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:15.906245 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:15.906245 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:15.906245 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:15.906245 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:15.906245 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:15.906245 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:15.906245 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:15.906245 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:15.906245 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 23:31:15.906245 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 23:31:15.906245 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 23:31:15.906245 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 23:31:15.907483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:15.907483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:15.907483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 23:31:15.907483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:15.907483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 23:31:15.907483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:15.907483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:15.907483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:15.907483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:15.907483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:15.907483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:15.907483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:15.907483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:15.907483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:15.907483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:15.907483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:15.907483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:15.907483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:15.908558 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:15.908558 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:15.908558 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 23:31:15.908558 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 23:31:15.908558 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 23:31:15.908558 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 23:31:15.908558 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 23:31:15.908558 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 23:31:15.908558 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 17 23:31:15.908558 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:15.909102 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6211a755-be9e-430d-93f9-576372ffa74d None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:15.917176 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 326/661] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 17 23:31:15 2022] GET /identity/v3/users/neutron => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:15.917176 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce515742-4ad4-4997-82d8-d19af29ee1ec None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:15.956052 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ede685-a392-4c3a-a432-8f0a6a5421f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:15.957058 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ede685-a392-4c3a-a432-8f0a6a5421f0 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:15.957544 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ede685-a392-4c3a-a432-8f0a6a5421f0 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:15.958032 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ede685-a392-4c3a-a432-8f0a6a5421f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:16.000736 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:16.028648 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce515742-4ad4-4997-82d8-d19af29ee1ec None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:16.039545 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ede685-a392-4c3a-a432-8f0a6a5421f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:16.041036 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ede685-a392-4c3a-a432-8f0a6a5421f0 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:16.066981 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e8b62-bf43-4c84-90b2-232e0511a98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:16.067830 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e8b62-bf43-4c84-90b2-232e0511a98d None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:16.068143 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e8b62-bf43-4c84-90b2-232e0511a98d None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:16.068416 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e8b62-bf43-4c84-90b2-232e0511a98d None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:16.069063 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23e8b62-bf43-4c84-90b2-232e0511a98d None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:16.069372 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23e8b62-bf43-4c84-90b2-232e0511a98d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:16.070731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23e8b62-bf43-4c84-90b2-232e0511a98d None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:16.087124 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 327/662] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:16.087124 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e8b62-bf43-4c84-90b2-232e0511a98d None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:16.146034 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:16.156061 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 336/663] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:31:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:31:16.157271 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ede685-a392-4c3a-a432-8f0a6a5421f0 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:16.187579 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e8b62-bf43-4c84-90b2-232e0511a98d None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:16.223642 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1374a6-b70b-41aa-9e25-aa8dc700e484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:16.224411 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1374a6-b70b-41aa-9e25-aa8dc700e484 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:16.224763 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1374a6-b70b-41aa-9e25-aa8dc700e484 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:16.225124 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1374a6-b70b-41aa-9e25-aa8dc700e484 None admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:16.225929 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1374a6-b70b-41aa-9e25-aa8dc700e484 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:16.226300 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1374a6-b70b-41aa-9e25-aa8dc700e484 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:16.227810 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1374a6-b70b-41aa-9e25-aa8dc700e484 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:16.231862 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:16.251721 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 328/664] 10.222.0.26 () {56 vars in 1139 bytes} [Wed Aug 17 23:31:16 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:16.251721 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1374a6-b70b-41aa-9e25-aa8dc700e484 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:16.270729 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ede685-a392-4c3a-a432-8f0a6a5421f0 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:16.306595 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24edda27-4010-45df-a627-82a245c35092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:16.307400 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24edda27-4010-45df-a627-82a245c35092 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:16.308244 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24edda27-4010-45df-a627-82a245c35092 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:16.308592 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24edda27-4010-45df-a627-82a245c35092 None admin] PATH_INFO: `/v3/roles/service` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:16.318866 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24edda27-4010-45df-a627-82a245c35092 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:31:16.323746 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-24edda27-4010-45df-a627-82a245c35092 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:31:16.323746 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:16.323746 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:16.323746 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:16.323746 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:16.323746 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:16.323746 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:16.323746 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:16.323746 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:16.323746 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:16.323746 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:16.323746 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:16.323746 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:31:16.323746 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:31:16.323746 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:31:16.323746 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:31:16.324950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:31:16.324950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:31:16.324950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:31:16.324950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:31:16.324950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:16.324950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:16.324950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:16.324950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:16.324950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:16.324950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:16.324950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:16.324950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:16.324950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:16.324950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:16.324950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:16.324950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:16.330830 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:16.330830 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:16.330830 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:16.330830 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:16.330830 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:31:16.330830 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:31:16.330830 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:31:16.330830 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:31:16.330830 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:31:16.330830 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:31:16.330830 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:31:16.330830 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:16.330830 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 337/665] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 17 23:31:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:16.330830 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24edda27-4010-45df-a627-82a245c35092 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:16.378105 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1374a6-b70b-41aa-9e25-aa8dc700e484 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:16.387050 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:16.427297 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd3a39-d788-4362-b891-687b7831d715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:16.428082 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd3a39-d788-4362-b891-687b7831d715 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:16.428384 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd3a39-d788-4362-b891-687b7831d715 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:16.428664 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd3a39-d788-4362-b891-687b7831d715 None admin] PATH_INFO: `/v3/projects/service` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:16.438319 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24edda27-4010-45df-a627-82a245c35092 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:16.439665 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd3a39-d788-4362-b891-687b7831d715 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:16.441184 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd3a39-d788-4362-b891-687b7831d715 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:16.462221 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-65dd3a39-d788-4362-b891-687b7831d715 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:16.462221 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:16.462221 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:16.462221 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:16.462221 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:16.462221 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:16.462221 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:16.462221 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:16.462221 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:16.462221 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:16.462221 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:16.462221 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:16.462221 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 23:31:16.462221 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 23:31:16.462221 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 23:31:16.464373 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 23:31:16.464373 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:16.464373 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:16.464373 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:16.464373 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:16.464373 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:16.464373 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:16.464373 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:16.464373 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:16.464373 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:16.464373 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:16.464373 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:16.464373 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:16.464373 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:16.464373 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:16.464373 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:16.464373 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:16.465304 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 23:31:16.465304 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 23:31:16.465304 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 23:31:16.465304 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 23:31:16.465304 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 23:31:16.465304 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 23:31:16.465304 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:16.465304 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:16.465304 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 329/666] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:31:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:16.465971 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd3a39-d788-4362-b891-687b7831d715 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:16.489660 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4daae63-b272-4913-b8d3-3b75c24f2240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:16.490222 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4daae63-b272-4913-b8d3-3b75c24f2240 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:16.490508 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4daae63-b272-4913-b8d3-3b75c24f2240 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:16.490785 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4daae63-b272-4913-b8d3-3b75c24f2240 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:16.501941 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4daae63-b272-4913-b8d3-3b75c24f2240 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:16.503418 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4daae63-b272-4913-b8d3-3b75c24f2240 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:16.519599 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-b4daae63-b272-4913-b8d3-3b75c24f2240 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:16.519599 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:16.519599 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:16.519599 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:16.519599 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:16.519599 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:16.519599 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:16.519599 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:16.519599 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:16.519599 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:16.519599 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:16.519599 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:16.519599 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:16.519599 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:16.519599 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:16.519599 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:16.520789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:16.520789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:16.520789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:16.520789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:16.520789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:16.520789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:16.520789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:16.520789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:16.520789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:16.520789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:16.520789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:16.520789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:16.520789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:16.520789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:16.520789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:16.520789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:16.520789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:16.531336 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:16.531336 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:16.531336 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:16.531336 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 338/667] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:16.531336 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4daae63-b272-4913-b8d3-3b75c24f2240 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:16.535602 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:16.590233 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd3a39-d788-4362-b891-687b7831d715 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:16.630176 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b2841-add7-47a6-9ce9-84b2c854ef96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:16.630784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b2841-add7-47a6-9ce9-84b2c854ef96 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:16.631068 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b2841-add7-47a6-9ce9-84b2c854ef96 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:16.631411 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b2841-add7-47a6-9ce9-84b2c854ef96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:16.659265 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4daae63-b272-4913-b8d3-3b75c24f2240 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:16.671656 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:16.703992 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0217ccb-bed9-4324-96a4-ebdd0eaf8d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:16.705043 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0217ccb-bed9-4324-96a4-ebdd0eaf8d8d None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:16.705520 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0217ccb-bed9-4324-96a4-ebdd0eaf8d8d None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:16.705991 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0217ccb-bed9-4324-96a4-ebdd0eaf8d8d None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:16.706953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07b2841-add7-47a6-9ce9-84b2c854ef96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:16.707175 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0217ccb-bed9-4324-96a4-ebdd0eaf8d8d None admin] RBAC: Adding query filter params (name=service) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:16.707708 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0217ccb-bed9-4324-96a4-ebdd0eaf8d8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:16.708462 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07b2841-add7-47a6-9ce9-84b2c854ef96 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:16.710287 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0217ccb-bed9-4324-96a4-ebdd0eaf8d8d None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:16.728574 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 339/668] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 17 23:31:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:16.730632 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0217ccb-bed9-4324-96a4-ebdd0eaf8d8d None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:16.781480 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:16.834809 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 330/669] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:31:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:31:16.834809 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b2841-add7-47a6-9ce9-84b2c854ef96 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:16.884886 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0217ccb-bed9-4324-96a4-ebdd0eaf8d8d None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:16.921735 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e452e4-5a6f-42f3-922f-18024a1f8cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:16.922351 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e452e4-5a6f-42f3-922f-18024a1f8cc4 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:16.922640 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e452e4-5a6f-42f3-922f-18024a1f8cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:16.922912 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e452e4-5a6f-42f3-922f-18024a1f8cc4 None admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:16.923611 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e452e4-5a6f-42f3-922f-18024a1f8cc4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:16.923918 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e452e4-5a6f-42f3-922f-18024a1f8cc4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:16.925378 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e452e4-5a6f-42f3-922f-18024a1f8cc4 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:16.939670 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 340/670] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 17 23:31:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:16.940357 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e452e4-5a6f-42f3-922f-18024a1f8cc4 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:16.952243 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b2841-add7-47a6-9ce9-84b2c854ef96 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:16.972243 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:16.987645 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe6ea0b-a64a-493b-a832-3a0e2856d8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:16.988276 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6ea0b-a64a-493b-a832-3a0e2856d8e3 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:16.988550 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6ea0b-a64a-493b-a832-3a0e2856d8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:16.988814 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6ea0b-a64a-493b-a832-3a0e2856d8e3 None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:16.989445 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe6ea0b-a64a-493b-a832-3a0e2856d8e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:16.989749 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe6ea0b-a64a-493b-a832-3a0e2856d8e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:16.991106 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe6ea0b-a64a-493b-a832-3a0e2856d8e3 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:17.002912 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 331/671] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:17.003487 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe6ea0b-a64a-493b-a832-3a0e2856d8e3 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:17.021758 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:17.043261 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e452e4-5a6f-42f3-922f-18024a1f8cc4 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:17.077218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb6103e-4294-459f-b2d0-9e6d9419c4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:17.077833 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6103e-4294-459f-b2d0-9e6d9419c4a0 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:17.078110 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6103e-4294-459f-b2d0-9e6d9419c4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:17.078380 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6103e-4294-459f-b2d0-9e6d9419c4a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:17.087992 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb6103e-4294-459f-b2d0-9e6d9419c4a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:17.089975 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb6103e-4294-459f-b2d0-9e6d9419c4a0 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:17.103776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-8fb6103e-4294-459f-b2d0-9e6d9419c4a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:17.103776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:17.103776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:17.103776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:17.103776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:17.103776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:17.103776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:17.103776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:17.103776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:17.103776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:17.103776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:17.103776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:17.103776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:17.103776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:17.103776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:17.103776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:17.109736 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:17.109736 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:17.109736 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:17.109736 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:17.109736 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:17.109736 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:17.109736 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:17.109736 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:17.109736 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:17.109736 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:17.109736 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:17.109736 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:17.109736 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:17.109736 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:17.109736 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:17.109736 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:17.109736 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:17.110696 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:17.110696 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:17.110696 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:17.110957 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe6ea0b-a64a-493b-a832-3a0e2856d8e3 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:17.111033 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 341/672] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:17.111033 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb6103e-4294-459f-b2d0-9e6d9419c4a0 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:17.139514 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd3b485-21a5-4fca-8f2d-5be395e30aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:17.140220 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3b485-21a5-4fca-8f2d-5be395e30aa7 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:17.140502 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3b485-21a5-4fca-8f2d-5be395e30aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:17.140773 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3b485-21a5-4fca-8f2d-5be395e30aa7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:17.151136 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd3b485-21a5-4fca-8f2d-5be395e30aa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:17.152907 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd3b485-21a5-4fca-8f2d-5be395e30aa7 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:17.166741 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-2bd3b485-21a5-4fca-8f2d-5be395e30aa7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:17.166741 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:17.166741 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:17.166741 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:17.166741 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:17.166741 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:17.166741 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:17.166741 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:17.166741 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:17.166741 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:17.166741 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:17.166741 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:17.166741 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:17.166741 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:17.166741 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:17.166741 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:17.172248 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:17.172248 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:17.172248 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:17.172248 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:17.172248 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:17.172248 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:17.172248 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:17.172248 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:17.172248 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:17.172248 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:17.172248 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:17.172248 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:17.172248 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:17.172248 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:17.172248 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:17.172248 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:17.172248 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:17.173391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:17.173391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:17.173391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:17.173391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 332/673] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:17.173391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc0fdf-04e4-4a30-be74-482fb189f7c8 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:17.173391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc0fdf-04e4-4a30-be74-482fb189f7c8 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:17.173391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc0fdf-04e4-4a30-be74-482fb189f7c8 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:17.173391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 333/674] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:17.173391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc0fdf-04e4-4a30-be74-482fb189f7c8 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:17.193356 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:17.210203 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb6103e-4294-459f-b2d0-9e6d9419c4a0 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:17.238959 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:17.249225 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66f518d-6be1-43cd-b56d-da6418f14ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:17.249835 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f518d-6be1-43cd-b56d-da6418f14ec2 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:17.250122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f518d-6be1-43cd-b56d-da6418f14ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:17.250399 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f518d-6be1-43cd-b56d-da6418f14ec2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:17.251124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66f518d-6be1-43cd-b56d-da6418f14ec2 None admin] RBAC: Adding query filter params (role.id=b64f80e6f3ea4ad49b94acdde42ccd87, scope.project.id=4de86915ef4340248ba8a8f51d56594e, user.id=62906d50b6e247c59b69809d3959d4de) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:17.251432 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66f518d-6be1-43cd-b56d-da6418f14ec2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:17.252901 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66f518d-6be1-43cd-b56d-da6418f14ec2 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:17.258716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 342/675] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:31:17 2022] GET /identity/v3/role_assignments?user.id=62906d50b6e247c59b69809d3959d4de&scope.project.id=4de86915ef4340248ba8a8f51d56594e&role.id=b64f80e6f3ea4ad49b94acdde42ccd87 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:17.259490 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66f518d-6be1-43cd-b56d-da6418f14ec2 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:17.282339 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc0fdf-04e4-4a30-be74-482fb189f7c8 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:17.324100 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e8fa8-dbc0-44fd-a7fd-69501e86ec6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:17.325079 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e8fa8-dbc0-44fd-a7fd-69501e86ec6e None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:17.325530 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e8fa8-dbc0-44fd-a7fd-69501e86ec6e None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:17.325944 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e8fa8-dbc0-44fd-a7fd-69501e86ec6e None admin] PATH_INFO: `/v3/projects/service` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:17.335026 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85e8fa8-dbc0-44fd-a7fd-69501e86ec6e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:17.336819 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85e8fa8-dbc0-44fd-a7fd-69501e86ec6e None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:17.349416 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-c85e8fa8-dbc0-44fd-a7fd-69501e86ec6e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:17.349416 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:17.349416 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:17.349416 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:17.349416 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:17.349416 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:17.349416 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:17.349416 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:17.349416 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:17.349416 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:17.349416 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:17.349416 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:17.349416 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 23:31:17.349416 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 23:31:17.349416 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 23:31:17.350454 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 23:31:17.350454 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:17.350454 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:17.350454 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:17.350454 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:17.350454 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:17.350454 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:17.350454 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:17.350454 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:17.350454 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:17.350454 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:17.350454 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:17.350454 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:17.350454 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:17.350454 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:17.350454 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:17.350454 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:17.351541 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 23:31:17.351541 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 23:31:17.351541 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 23:31:17.351541 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 23:31:17.351541 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 23:31:17.351541 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 23:31:17.351541 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:17.351541 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:17.358147 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 334/676] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:31:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:17.359368 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e8fa8-dbc0-44fd-a7fd-69501e86ec6e None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:17.370556 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66f518d-6be1-43cd-b56d-da6418f14ec2 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:17.411833 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445b005-cc86-456c-bf37-13ad90ef2fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:17.412475 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445b005-cc86-456c-bf37-13ad90ef2fa4 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:17.412754 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445b005-cc86-456c-bf37-13ad90ef2fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:17.413024 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445b005-cc86-456c-bf37-13ad90ef2fa4 None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:17.413673 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d445b005-cc86-456c-bf37-13ad90ef2fa4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:17.413975 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d445b005-cc86-456c-bf37-13ad90ef2fa4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:17.415693 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d445b005-cc86-456c-bf37-13ad90ef2fa4 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:17.429156 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 343/677] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:17.430381 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa66a64-1c28-47b7-bea4-be93a90796a8 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:17.430671 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa66a64-1c28-47b7-bea4-be93a90796a8 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:17.431727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa66a64-1c28-47b7-bea4-be93a90796a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:17.454977 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-dfa66a64-1c28-47b7-bea4-be93a90796a8 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:17.456218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-dfa66a64-1c28-47b7-bea4-be93a90796a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:17.458395 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:17.479947 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e8fa8-dbc0-44fd-a7fd-69501e86ec6e None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:17.508720 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:17.520071 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f3fde3-a434-4801-bc13-4627b7b69551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:17.520679 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3fde3-a434-4801-bc13-4627b7b69551 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:17.520972 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3fde3-a434-4801-bc13-4627b7b69551 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:17.521224 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3fde3-a434-4801-bc13-4627b7b69551 None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:17.521871 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f3fde3-a434-4801-bc13-4627b7b69551 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:17.522173 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f3fde3-a434-4801-bc13-4627b7b69551 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:17.523545 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f3fde3-a434-4801-bc13-4627b7b69551 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:17.529584 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 344/678] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:31:17.530225 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa66a64-1c28-47b7-bea4-be93a90796a8 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:17.552281 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 335/679] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:17.553163 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f3fde3-a434-4801-bc13-4627b7b69551 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:17.663985 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f3fde3-a434-4801-bc13-4627b7b69551 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:17.665715 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa66a64-1c28-47b7-bea4-be93a90796a8 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:17.698886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dbaba5-db0b-4be0-9b1c-e205bd1e5352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:17.699568 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbaba5-db0b-4be0-9b1c-e205bd1e5352 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:17.699846 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbaba5-db0b-4be0-9b1c-e205bd1e5352 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:17.700145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbaba5-db0b-4be0-9b1c-e205bd1e5352 None admin] PATH_INFO: `/v3/users/swift` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:17.705886 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab67de3-bde2-4ae3-8c88-78ea1bf82207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:17.706738 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab67de3-bde2-4ae3-8c88-78ea1bf82207 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:17.707237 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab67de3-bde2-4ae3-8c88-78ea1bf82207 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:17.707706 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab67de3-bde2-4ae3-8c88-78ea1bf82207 None admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:17.708488 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dbaba5-db0b-4be0-9b1c-e205bd1e5352 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:17.709030 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab67de3-bde2-4ae3-8c88-78ea1bf82207 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:17.709566 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab67de3-bde2-4ae3-8c88-78ea1bf82207 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:17.710244 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dbaba5-db0b-4be0-9b1c-e205bd1e5352 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:17.711597 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab67de3-bde2-4ae3-8c88-78ea1bf82207 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:17.719387 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:17.722780 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-86dbaba5-db0b-4be0-9b1c-e205bd1e5352 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 17 23:31:17.722780 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:17.722780 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:17.722780 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:17.722780 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:17.722780 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:17.722780 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:17.722780 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:17.722780 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:17.722780 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:17.722780 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:17.722780 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:17.722780 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 23:31:17.722780 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 23:31:17.722780 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 23:31:17.722780 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 23:31:17.723736 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:17.723736 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:17.723736 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 23:31:17.723736 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:17.723736 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 23:31:17.723736 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:17.723736 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:17.723736 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:17.723736 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:17.723736 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:17.723736 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:17.723736 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:17.723736 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:17.723736 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:17.723736 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:17.723736 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:17.723736 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:17.723736 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:17.724742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:17.724742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:17.724742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 23:31:17.724742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 23:31:17.724742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 23:31:17.724742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 23:31:17.724742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 23:31:17.724742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 23:31:17.724742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 17 23:31:17.724742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:17.729591 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 345/681] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 17 23:31:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:17.729733 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 336/681] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 17 23:31:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:17.730582 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dbaba5-db0b-4be0-9b1c-e205bd1e5352 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:17.731067 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab67de3-bde2-4ae3-8c88-78ea1bf82207 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:17.778622 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:17.835745 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dbaba5-db0b-4be0-9b1c-e205bd1e5352 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:17.843175 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab67de3-bde2-4ae3-8c88-78ea1bf82207 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:17.875045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd944e85-b1bc-49a5-ac4d-9cd581cd5c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:17.875667 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd944e85-b1bc-49a5-ac4d-9cd581cd5c83 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:17.875954 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd944e85-b1bc-49a5-ac4d-9cd581cd5c83 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:17.876250 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd944e85-b1bc-49a5-ac4d-9cd581cd5c83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:17.878581 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e259b26a-e3b7-4286-8074-10f5267c4f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:17.879173 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259b26a-e3b7-4286-8074-10f5267c4f30 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:17.879458 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259b26a-e3b7-4286-8074-10f5267c4f30 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:17.879722 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259b26a-e3b7-4286-8074-10f5267c4f30 None admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:17.880472 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e259b26a-e3b7-4286-8074-10f5267c4f30 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:17.880778 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e259b26a-e3b7-4286-8074-10f5267c4f30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:17.882334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e259b26a-e3b7-4286-8074-10f5267c4f30 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:17.883192 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd944e85-b1bc-49a5-ac4d-9cd581cd5c83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:17.885338 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd944e85-b1bc-49a5-ac4d-9cd581cd5c83 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:17.896519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-bd944e85-b1bc-49a5-ac4d-9cd581cd5c83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:17.896519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:17.896519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:17.896519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:17.896519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:17.896519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:17.896519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:17.896519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:17.896519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:17.896519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:17.896519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:17.896519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:17.896519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 23:31:17.896519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 23:31:17.896519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 23:31:17.897635 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 23:31:17.897635 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:17.897635 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:17.897635 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:17.897635 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:17.897635 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:17.897635 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:17.897635 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:17.897635 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:17.897635 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:17.897635 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:17.897635 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:17.897635 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:17.897635 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:17.897635 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:17.897635 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:17.897635 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:17.898531 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 23:31:17.898531 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 23:31:17.898531 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 23:31:17.898531 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 23:31:17.898531 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 23:31:17.898531 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 23:31:17.898531 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:17.898531 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:17.898531 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 337/682] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:31:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:17.898531 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd944e85-b1bc-49a5-ac4d-9cd581cd5c83 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:17.908407 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 346/683] 10.222.0.26 () {56 vars in 1135 bytes} [Wed Aug 17 23:31:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:17.908407 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e259b26a-e3b7-4286-8074-10f5267c4f30 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:17.972089 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:18.011305 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd944e85-b1bc-49a5-ac4d-9cd581cd5c83 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:18.018282 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:18.020970 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e259b26a-e3b7-4286-8074-10f5267c4f30 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:18.047576 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3734c-2f88-4d5b-bf9e-8eb1642235c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:18.048365 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3734c-2f88-4d5b-bf9e-8eb1642235c8 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:18.048737 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3734c-2f88-4d5b-bf9e-8eb1642235c8 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:18.049109 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3734c-2f88-4d5b-bf9e-8eb1642235c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:18.049899 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3734c-2f88-4d5b-bf9e-8eb1642235c8 None admin] RBAC: Adding query filter params (role.id=b64f80e6f3ea4ad49b94acdde42ccd87, scope.project.id=4de86915ef4340248ba8a8f51d56594e, user.id=ec1dcbe43b0749c3bcc4af9100d771d1) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:18.050203 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3734c-2f88-4d5b-bf9e-8eb1642235c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:18.051616 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3734c-2f88-4d5b-bf9e-8eb1642235c8 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:18.067999 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 338/684] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:31:17 2022] GET /identity/v3/role_assignments?user.id=ec1dcbe43b0749c3bcc4af9100d771d1&scope.project.id=4de86915ef4340248ba8a8f51d56594e&role.id=b64f80e6f3ea4ad49b94acdde42ccd87 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:18.067999 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3734c-2f88-4d5b-bf9e-8eb1642235c8 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:18.071365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972eee66-ab25-4664-a600-a651810d6d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:18.072143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972eee66-ab25-4664-a600-a651810d6d69 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:18.072500 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972eee66-ab25-4664-a600-a651810d6d69 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:18.072911 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972eee66-ab25-4664-a600-a651810d6d69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:18.084863 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972eee66-ab25-4664-a600-a651810d6d69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:18.086591 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972eee66-ab25-4664-a600-a651810d6d69 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:18.107652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-972eee66-ab25-4664-a600-a651810d6d69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:18.107652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:18.107652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:18.107652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:18.107652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:18.107652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:18.107652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:18.107652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:18.107652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:18.107652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:18.107652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:18.107652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:18.107652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:18.107652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:18.107652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:18.107652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:18.108789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:18.108789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:18.108789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:18.108789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:18.108789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:18.108789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:18.108789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:18.108789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:18.108789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:18.108789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:18.108789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:18.108789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:18.108789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:18.108789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:18.108789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:18.108789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:18.108789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:18.109682 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:18.109682 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:18.109682 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:18.114396 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 347/685] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:18.115409 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972eee66-ab25-4664-a600-a651810d6d69 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:18.208880 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3734c-2f88-4d5b-bf9e-8eb1642235c8 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:18.233316 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972eee66-ab25-4664-a600-a651810d6d69 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:18.247361 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:18.253218 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b277719-6ad9-4cb9-937f-a6c58babbd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:18.253830 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b277719-6ad9-4cb9-937f-a6c58babbd86 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:18.254104 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b277719-6ad9-4cb9-937f-a6c58babbd86 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:18.255062 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b277719-6ad9-4cb9-937f-a6c58babbd86 None admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:18.256226 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b277719-6ad9-4cb9-937f-a6c58babbd86 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:18.256226 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b277719-6ad9-4cb9-937f-a6c58babbd86 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:18.257649 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b277719-6ad9-4cb9-937f-a6c58babbd86 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:18.278091 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 339/686] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 17 23:31:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:18.278806 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b277719-6ad9-4cb9-937f-a6c58babbd86 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:18.286320 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb5823-f8fc-46ad-abb1-2f9d1b4ab711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:18.286960 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb5823-f8fc-46ad-abb1-2f9d1b4ab711 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:18.287245 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb5823-f8fc-46ad-abb1-2f9d1b4ab711 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:18.287522 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb5823-f8fc-46ad-abb1-2f9d1b4ab711 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:18.317925 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:18.375670 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bb5823-f8fc-46ad-abb1-2f9d1b4ab711 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:18.377287 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bb5823-f8fc-46ad-abb1-2f9d1b4ab711 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:18.399809 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b277719-6ad9-4cb9-937f-a6c58babbd86 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:18.443812 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c3215a-3965-414c-a5bd-059ddc4f7716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:18.444449 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3215a-3965-414c-a5bd-059ddc4f7716 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:18.444729 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3215a-3965-414c-a5bd-059ddc4f7716 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:18.445018 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3215a-3965-414c-a5bd-059ddc4f7716 None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:18.445689 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c3215a-3965-414c-a5bd-059ddc4f7716 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:18.446129 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c3215a-3965-414c-a5bd-059ddc4f7716 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:18.448053 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c3215a-3965-414c-a5bd-059ddc4f7716 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:18.463722 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 340/687] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:18.464611 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c3215a-3965-414c-a5bd-059ddc4f7716 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:18.504217 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 348/688] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:31:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:31:18.505462 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb5823-f8fc-46ad-abb1-2f9d1b4ab711 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:18.516530 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:18.572022 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:18.589651 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c3215a-3965-414c-a5bd-059ddc4f7716 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:18.632741 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e902f8-dd56-4358-be3a-d2164f26d9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:18.633350 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e902f8-dd56-4358-be3a-d2164f26d9a5 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:18.633630 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e902f8-dd56-4358-be3a-d2164f26d9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:18.633900 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e902f8-dd56-4358-be3a-d2164f26d9a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:18.634622 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e902f8-dd56-4358-be3a-d2164f26d9a5 None admin] RBAC: Adding query filter params (role.id=b64f80e6f3ea4ad49b94acdde42ccd87, scope.project.id=4de86915ef4340248ba8a8f51d56594e, user.id=16e2758f1cd144a98b98d9be0fa46ae2) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:18.634926 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e902f8-dd56-4358-be3a-d2164f26d9a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:18.636489 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e902f8-dd56-4358-be3a-d2164f26d9a5 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:18.647999 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 341/689] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:31:18 2022] GET /identity/v3/role_assignments?user.id=16e2758f1cd144a98b98d9be0fa46ae2&scope.project.id=4de86915ef4340248ba8a8f51d56594e&role.id=b64f80e6f3ea4ad49b94acdde42ccd87 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:18.647999 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e902f8-dd56-4358-be3a-d2164f26d9a5 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:18.661335 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb5823-f8fc-46ad-abb1-2f9d1b4ab711 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:18.733059 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36809ae-f957-446d-83ab-7eacdc753f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:18.733680 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36809ae-f957-446d-83ab-7eacdc753f00 None admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:18.734004 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36809ae-f957-446d-83ab-7eacdc753f00 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:18.734281 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36809ae-f957-446d-83ab-7eacdc753f00 None admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:18.734935 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36809ae-f957-446d-83ab-7eacdc753f00 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:18.736441 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36809ae-f957-446d-83ab-7eacdc753f00 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:18.775066 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 349/690] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Aug 17 23:31:18 2022] POST /identity/v3/users => generated 254 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:31:18.788365 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e902f8-dd56-4358-be3a-d2164f26d9a5 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:18.828375 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b3e9be-4c57-4d53-bafe-16a20bd05849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:18.829021 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3e9be-4c57-4d53-bafe-16a20bd05849 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:18.829303 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3e9be-4c57-4d53-bafe-16a20bd05849 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:18.829558 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3e9be-4c57-4d53-bafe-16a20bd05849 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:18.842318 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b3e9be-4c57-4d53-bafe-16a20bd05849 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:18.843779 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b3e9be-4c57-4d53-bafe-16a20bd05849 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:18.860500 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-49b3e9be-4c57-4d53-bafe-16a20bd05849 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:18.860500 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:18.860500 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:18.860500 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:18.860500 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:18.860500 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:18.860500 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:18.860500 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:18.860500 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:18.860500 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:18.860500 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:18.860500 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:18.860500 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:18.860500 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:18.860500 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:18.860500 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:18.861517 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:18.861517 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:18.861517 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:18.861517 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:18.861517 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:18.861517 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:18.861517 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:18.861517 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:18.861517 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:18.861517 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:18.861517 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:18.861517 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:18.861517 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:18.861517 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:18.861517 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:18.861517 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:18.861517 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:18.862833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:18.862833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:18.862833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:18.862833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 342/691] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:18.867562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36809ae-f957-446d-83ab-7eacdc753f00 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:18.943690 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:18.988886 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36809ae-f957-446d-83ab-7eacdc753f00 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:19.026588 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31514507-2d0f-49be-9630-04f8c9d33105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:19.027115 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31514507-2d0f-49be-9630-04f8c9d33105 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:19.027386 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31514507-2d0f-49be-9630-04f8c9d33105 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:19.028004 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31514507-2d0f-49be-9630-04f8c9d33105 None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:19.028682 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31514507-2d0f-49be-9630-04f8c9d33105 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:19.028981 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31514507-2d0f-49be-9630-04f8c9d33105 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:19.030339 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31514507-2d0f-49be-9630-04f8c9d33105 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:19.043023 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 350/692] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:19.048668 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b3e9be-4c57-4d53-bafe-16a20bd05849 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:19.058987 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:19.161066 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b3e9be-4c57-4d53-bafe-16a20bd05849 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:19.199998 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4cb4db-445d-4199-8080-8febd216686f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:19.200714 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4cb4db-445d-4199-8080-8febd216686f None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:19.200984 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4cb4db-445d-4199-8080-8febd216686f None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:19.201246 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4cb4db-445d-4199-8080-8febd216686f None admin] PATH_INFO: `/v3/projects/service` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:19.208504 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4cb4db-445d-4199-8080-8febd216686f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:19.210013 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4cb4db-445d-4199-8080-8febd216686f None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:19.222974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-3d4cb4db-445d-4199-8080-8febd216686f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:19.222974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:19.222974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:19.222974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:19.222974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:19.222974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:19.222974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:19.222974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:19.222974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:19.222974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:19.222974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:19.222974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:19.222974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 23:31:19.222974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 23:31:19.222974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 23:31:19.223886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 23:31:19.223886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:19.223886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:19.223886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:19.223886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:19.223886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:19.223886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:19.223886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:19.223886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:19.223886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:19.223886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:19.223886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:19.223886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:19.223886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:19.223886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:19.223886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:19.223886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:19.225259 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 23:31:19.225259 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 23:31:19.225259 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 23:31:19.225259 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 23:31:19.225259 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 23:31:19.225259 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 23:31:19.225259 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:19.225259 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:19.225259 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 343/693] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:31:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:19.230510 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31514507-2d0f-49be-9630-04f8c9d33105 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:19.360608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31514507-2d0f-49be-9630-04f8c9d33105 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:19.390819 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524ecc4a-e5c6-4809-b0ca-71cd037b009a None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:19.391154 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524ecc4a-e5c6-4809-b0ca-71cd037b009a None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:19.391372 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524ecc4a-e5c6-4809-b0ca-71cd037b009a None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:19.393003 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 344/694] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 23:31:19.398090 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac513a9d-59bb-4b89-9d8a-550f239424b1 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:19.398373 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac513a9d-59bb-4b89-9d8a-550f239424b1 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:19.398649 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac513a9d-59bb-4b89-9d8a-550f239424b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:19.402338 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:19.427492 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c54eee-924f-4c71-8dab-435b02a430d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:19.428414 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c54eee-924f-4c71-8dab-435b02a430d1 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:19.428868 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c54eee-924f-4c71-8dab-435b02a430d1 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:19.429314 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c54eee-924f-4c71-8dab-435b02a430d1 None admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:19.430204 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c54eee-924f-4c71-8dab-435b02a430d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:19.430682 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c54eee-924f-4c71-8dab-435b02a430d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:19.432873 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c54eee-924f-4c71-8dab-435b02a430d1 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:19.434022 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-ac513a9d-59bb-4b89-9d8a-550f239424b1 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:19.435389 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-ac513a9d-59bb-4b89-9d8a-550f239424b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:19.457947 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 351/695] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 17 23:31:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:19.465325 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c54eee-924f-4c71-8dab-435b02a430d1 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:19.565437 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:19.579452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 345/696] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 23:31:19.614804 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c54eee-924f-4c71-8dab-435b02a430d1 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:19.649841 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30594ea2-bc1d-4ee4-a354-49c10966e0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:19.650537 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30594ea2-bc1d-4ee4-a354-49c10966e0c7 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:19.650909 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30594ea2-bc1d-4ee4-a354-49c10966e0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:19.651262 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30594ea2-bc1d-4ee4-a354-49c10966e0c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:19.652082 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30594ea2-bc1d-4ee4-a354-49c10966e0c7 None admin] RBAC: Adding query filter params (role.id=b64f80e6f3ea4ad49b94acdde42ccd87, scope.project.id=4de86915ef4340248ba8a8f51d56594e, user.id=ee422ea90557444fabdc9c292865b9b8) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:19.652453 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30594ea2-bc1d-4ee4-a354-49c10966e0c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:19.654206 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30594ea2-bc1d-4ee4-a354-49c10966e0c7 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:19.660666 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 352/697] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:31:19 2022] GET /identity/v3/role_assignments?user.id=ee422ea90557444fabdc9c292865b9b8&scope.project.id=4de86915ef4340248ba8a8f51d56594e&role.id=b64f80e6f3ea4ad49b94acdde42ccd87 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:19.956590 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac513a9d-59bb-4b89-9d8a-550f239424b1 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:20.061296 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac513a9d-59bb-4b89-9d8a-550f239424b1 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:20.097793 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43901b6d-0efd-4489-bcf6-a3a38596bd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:20.098428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43901b6d-0efd-4489-bcf6-a3a38596bd3d None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:20.098705 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43901b6d-0efd-4489-bcf6-a3a38596bd3d None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:20.098975 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43901b6d-0efd-4489-bcf6-a3a38596bd3d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:20.099566 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43901b6d-0efd-4489-bcf6-a3a38596bd3d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:20.100978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43901b6d-0efd-4489-bcf6-a3a38596bd3d None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:20.111834 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-43901b6d-0efd-4489-bcf6-a3a38596bd3d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 17 23:31:20.111834 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:20.111834 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 23:31:20.111834 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 23:31:20.111834 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 23:31:20.111834 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 23:31:20.111834 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 23:31:20.111834 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:20.111834 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 17 23:31:20.111834 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:20.111834 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 23:31:20.111834 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:20.111834 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:20.111834 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:20.111834 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:20.111834 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:20.111834 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:20.113045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:20.113045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:20.113045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:20.113045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:20.113045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:20.113045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:20.113045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 23:31:20.113045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 23:31:20.113045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 23:31:20.113045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 23:31:20.113045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:20.113045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:20.113045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:20.113045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:20.113045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:20.113045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:20.114408 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:20.114408 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:20.114408 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:20.114408 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:20.114408 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:20.114408 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:20.114408 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:20.114408 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:20.114408 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:20.114408 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:20.114408 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 23:31:20.114408 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:20.114408 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 17 23:31:20.114408 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:20.114408 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 346/698] 10.222.0.26 () {56 vars in 1129 bytes} [Wed Aug 17 23:31:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:20.119190 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30594ea2-bc1d-4ee4-a354-49c10966e0c7 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:20.181284 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb2964-e058-44c3-81d6-8eb1efee92e3 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:20.181655 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb2964-e058-44c3-81d6-8eb1efee92e3 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:20.181823 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb2964-e058-44c3-81d6-8eb1efee92e3 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:20.184101 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 347/699] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:20.186550 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:20.190884 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8a5e4-88fb-457a-98f5-cbf79e93e4b9 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:20.191205 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8a5e4-88fb-457a-98f5-cbf79e93e4b9 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:20.191439 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8a5e4-88fb-457a-98f5-cbf79e93e4b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:20.212918 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-0bc8a5e4-88fb-457a-98f5-cbf79e93e4b9 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:20.214763 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-0bc8a5e4-88fb-457a-98f5-cbf79e93e4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:20.262990 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30594ea2-bc1d-4ee4-a354-49c10966e0c7 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:20.288740 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:20.309606 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 348/700] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:31:20.313329 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89cd99-0978-4fd5-8b2a-a384af521011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:20.313898 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89cd99-0978-4fd5-8b2a-a384af521011 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:20.314180 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89cd99-0978-4fd5-8b2a-a384af521011 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:20.314454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89cd99-0978-4fd5-8b2a-a384af521011 None admin] PATH_INFO: `/v3/services` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:20.315093 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c89cd99-0978-4fd5-8b2a-a384af521011 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:20.315394 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c89cd99-0978-4fd5-8b2a-a384af521011 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:20.316785 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c89cd99-0978-4fd5-8b2a-a384af521011 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:20.327904 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 353/701] 10.222.0.26 () {56 vars in 1124 bytes} [Wed Aug 17 23:31:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:20.333126 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc8a5e4-88fb-457a-98f5-cbf79e93e4b9 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:20.456734 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc8a5e4-88fb-457a-98f5-cbf79e93e4b9 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:20.516185 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba8391-5612-45ab-8d4a-c1e8ee1f2a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:20.516799 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba8391-5612-45ab-8d4a-c1e8ee1f2a07 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:20.517077 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba8391-5612-45ab-8d4a-c1e8ee1f2a07 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:20.517348 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba8391-5612-45ab-8d4a-c1e8ee1f2a07 None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:20.517987 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba8391-5612-45ab-8d4a-c1e8ee1f2a07 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:20.518290 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba8391-5612-45ab-8d4a-c1e8ee1f2a07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:20.519665 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba8391-5612-45ab-8d4a-c1e8ee1f2a07 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:20.529105 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 349/702] 10.222.0.26 () {56 vars in 1124 bytes} [Wed Aug 17 23:31:20 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:31:20.666326 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89cd99-0978-4fd5-8b2a-a384af521011 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:20.788105 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89cd99-0978-4fd5-8b2a-a384af521011 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:20.802200 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:20.828588 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232687cb-7c7a-49f0-9442-94c5608e1b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:20.829231 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232687cb-7c7a-49f0-9442-94c5608e1b04 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:20.829511 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232687cb-7c7a-49f0-9442-94c5608e1b04 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:20.829786 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232687cb-7c7a-49f0-9442-94c5608e1b04 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:20.849463 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232687cb-7c7a-49f0-9442-94c5608e1b04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:31:20.855289 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-232687cb-7c7a-49f0-9442-94c5608e1b04 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:31:20.855289 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:20.855289 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:20.855289 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:20.855289 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:20.855289 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:20.855289 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:20.855289 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:20.855289 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:20.855289 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:20.855289 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:20.855289 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:20.855289 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:31:20.855289 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:31:20.855289 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:31:20.855289 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:31:20.856290 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:31:20.856290 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:31:20.856290 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:31:20.856290 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:31:20.856290 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:20.856290 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:20.856290 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:20.856290 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:20.856290 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:20.856290 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:20.856290 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:20.856290 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:20.856290 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:20.856290 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:20.856290 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:20.856290 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:20.857209 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:20.857209 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:20.857209 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:20.857209 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:20.857209 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:31:20.857209 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:31:20.857209 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:31:20.857209 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:31:20.857209 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:31:20.857209 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:31:20.857209 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:31:20.857209 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:20.857209 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 354/703] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 17 23:31:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:20.862401 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba8391-5612-45ab-8d4a-c1e8ee1f2a07 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:20.942410 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:21.000437 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba8391-5612-45ab-8d4a-c1e8ee1f2a07 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:21.050976 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616af305-b4ee-4773-9e26-6b77e65f5d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:21.051558 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616af305-b4ee-4773-9e26-6b77e65f5d21 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:21.051844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616af305-b4ee-4773-9e26-6b77e65f5d21 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:21.052135 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616af305-b4ee-4773-9e26-6b77e65f5d21 None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:21.052813 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616af305-b4ee-4773-9e26-6b77e65f5d21 None admin] RBAC: Adding query filter params (name=admin) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:21.053106 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616af305-b4ee-4773-9e26-6b77e65f5d21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:21.054472 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616af305-b4ee-4773-9e26-6b77e65f5d21 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:21.066250 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 350/704] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:31:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:21.071679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232687cb-7c7a-49f0-9442-94c5608e1b04 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:21.201202 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232687cb-7c7a-49f0-9442-94c5608e1b04 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:21.238147 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f6908-4761-46c3-b62a-3123be81fd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:21.239176 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f6908-4761-46c3-b62a-3123be81fd39 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:21.239678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f6908-4761-46c3-b62a-3123be81fd39 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:21.240206 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f6908-4761-46c3-b62a-3123be81fd39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:21.254369 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f6908-4761-46c3-b62a-3123be81fd39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:21.256229 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f6908-4761-46c3-b62a-3123be81fd39 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:21.271411 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-6c4f6908-4761-46c3-b62a-3123be81fd39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:21.271411 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:21.271411 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:21.271411 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:21.271411 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:21.271411 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:21.271411 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:21.271411 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:21.271411 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:21.271411 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:21.271411 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:21.271411 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:21.271411 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:21.271411 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:21.271411 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:21.271411 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:21.272711 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:21.272711 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:21.272711 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:21.272711 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:21.272711 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:21.272711 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:21.272711 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:21.272711 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:21.272711 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:21.272711 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:21.272711 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:21.272711 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:21.272711 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:21.272711 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:21.272711 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:21.272711 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:21.272711 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:21.273622 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:21.273622 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:21.273622 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:21.275648 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 355/705] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:21.284064 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616af305-b4ee-4773-9e26-6b77e65f5d21 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:21.285219 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3683af3-0341-47d1-a0f7-5aeb20e7ec5c None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:21.285504 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3683af3-0341-47d1-a0f7-5aeb20e7ec5c None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:21.285777 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3683af3-0341-47d1-a0f7-5aeb20e7ec5c None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:21.287371 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 356/706] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 23:31:21.293930 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a5c7e-8218-4b70-98c9-ff5fbd2ad496 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:21.294206 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a5c7e-8218-4b70-98c9-ff5fbd2ad496 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:21.294475 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a5c7e-8218-4b70-98c9-ff5fbd2ad496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:21.321496 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-101a5c7e-8218-4b70-98c9-ff5fbd2ad496 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:21.322627 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-101a5c7e-8218-4b70-98c9-ff5fbd2ad496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:21.344310 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:21.399241 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616af305-b4ee-4773-9e26-6b77e65f5d21 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:21.400500 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 357/707] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 23:31:21.402107 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b20bf-c360-45d3-8674-fed6bfeeeeae None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:21.402439 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b20bf-c360-45d3-8674-fed6bfeeeeae None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:21.402982 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b20bf-c360-45d3-8674-fed6bfeeeeae None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:21.404544 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 358/708] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:21.410463 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d4a0f-7cc5-46d1-9a68-1b5332023d5a None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:21.410747 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d4a0f-7cc5-46d1-9a68-1b5332023d5a None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:21.411020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d4a0f-7cc5-46d1-9a68-1b5332023d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:21.432084 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-700d4a0f-7cc5-46d1-9a68-1b5332023d5a None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:21.433834 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-700d4a0f-7cc5-46d1-9a68-1b5332023d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:21.442225 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:21.456587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40953ea-cbe5-4f36-a90b-e0ecf3a5afde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:21.457368 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40953ea-cbe5-4f36-a90b-e0ecf3a5afde None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:21.457789 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40953ea-cbe5-4f36-a90b-e0ecf3a5afde None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:21.458223 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40953ea-cbe5-4f36-a90b-e0ecf3a5afde None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:21.459028 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40953ea-cbe5-4f36-a90b-e0ecf3a5afde None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:21.459469 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40953ea-cbe5-4f36-a90b-e0ecf3a5afde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:21.461022 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40953ea-cbe5-4f36-a90b-e0ecf3a5afde None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:21.474700 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 351/709] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:21.485564 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40953ea-cbe5-4f36-a90b-e0ecf3a5afde None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:21.546041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 359/710] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:31:21.638699 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40953ea-cbe5-4f36-a90b-e0ecf3a5afde None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:21.682570 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bedab5-247e-4861-8c21-cd656f5c73e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:21.683197 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bedab5-247e-4861-8c21-cd656f5c73e5 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:21.683477 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bedab5-247e-4861-8c21-cd656f5c73e5 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:21.683747 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bedab5-247e-4861-8c21-cd656f5c73e5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:21.694906 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bedab5-247e-4861-8c21-cd656f5c73e5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:21.696458 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bedab5-247e-4861-8c21-cd656f5c73e5 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:21.711619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-81bedab5-247e-4861-8c21-cd656f5c73e5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 17 23:31:21.711619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:21.711619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:21.711619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:21.711619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:21.711619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:21.711619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:21.711619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:21.711619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:21.711619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:21.711619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:21.711619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:21.711619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 23:31:21.711619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 23:31:21.711619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 23:31:21.711619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 23:31:21.712603 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:21.712603 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:21.712603 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 23:31:21.712603 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:21.712603 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 23:31:21.712603 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:21.712603 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:21.712603 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:21.712603 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:21.712603 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:21.712603 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:21.712603 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:21.712603 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:21.712603 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:21.712603 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:21.712603 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:21.712603 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:21.712603 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:21.714111 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:21.714111 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:21.714111 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 23:31:21.714111 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 23:31:21.714111 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 23:31:21.714111 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 23:31:21.714111 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 23:31:21.714111 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 23:31:21.714111 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 17 23:31:21.714111 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:21.714111 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 352/711] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Aug 17 23:31:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:21.719315 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d4a0f-7cc5-46d1-9a68-1b5332023d5a None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:21.756984 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bedab5-247e-4861-8c21-cd656f5c73e5 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:21.798142 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:21.840045 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:21.863101 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bedab5-247e-4861-8c21-cd656f5c73e5 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:21.894247 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d4a0f-7cc5-46d1-9a68-1b5332023d5a None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:21.899225 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ea952-5e56-4f93-8048-b0434d092851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:21.899854 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ea952-5e56-4f93-8048-b0434d092851 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:21.900169 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ea952-5e56-4f93-8048-b0434d092851 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:21.900437 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ea952-5e56-4f93-8048-b0434d092851 None admin] PATH_INFO: `/v3/roles/service` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:21.912960 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967ea952-5e56-4f93-8048-b0434d092851 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:31:21.918665 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-967ea952-5e56-4f93-8048-b0434d092851 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:31:21.918665 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:21.918665 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:21.918665 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:21.918665 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:21.918665 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:21.918665 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:21.918665 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:21.918665 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:21.918665 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:21.918665 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:21.918665 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:21.918665 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:31:21.918665 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:31:21.918665 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:31:21.918665 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:31:21.919643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:31:21.919643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:31:21.919643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:31:21.919643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:31:21.919643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:21.919643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:21.919643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:21.919643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:21.919643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:21.919643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:21.919643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:21.919643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:21.919643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:21.919643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:21.919643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:21.919643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:21.920583 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:21.920583 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:21.920583 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:21.920583 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:21.920583 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:31:21.920583 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:31:21.920583 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:31:21.920583 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:31:21.920583 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:31:21.920583 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:31:21.920583 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:31:21.920583 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:21.924530 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 353/712] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 17 23:31:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:31:21.926149 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ea952-5e56-4f93-8048-b0434d092851 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:22.011265 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0556e3-6397-4421-a042-7fa6652edb59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:22.012341 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0556e3-6397-4421-a042-7fa6652edb59 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:22.012854 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0556e3-6397-4421-a042-7fa6652edb59 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:22.013426 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0556e3-6397-4421-a042-7fa6652edb59 None admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:22.014522 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0556e3-6397-4421-a042-7fa6652edb59 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:22.015081 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0556e3-6397-4421-a042-7fa6652edb59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:22.016842 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0556e3-6397-4421-a042-7fa6652edb59 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:22.032367 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:22.040677 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 360/713] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Aug 17 23:31:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:22.041837 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0556e3-6397-4421-a042-7fa6652edb59 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:22.042039 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ea952-5e56-4f93-8048-b0434d092851 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:22.087981 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a3cff-5e49-4293-8610-617546136bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:22.089384 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a3cff-5e49-4293-8610-617546136bb9 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:22.089683 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a3cff-5e49-4293-8610-617546136bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:22.089953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a3cff-5e49-4293-8610-617546136bb9 None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:22.090613 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a3cff-5e49-4293-8610-617546136bb9 None admin] RBAC: Adding query filter params (name=service) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:22.090912 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a3cff-5e49-4293-8610-617546136bb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:22.092660 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a3cff-5e49-4293-8610-617546136bb9 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:22.103948 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 354/714] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 17 23:31:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:31:22.110011 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a3cff-5e49-4293-8610-617546136bb9 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:22.155918 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0556e3-6397-4421-a042-7fa6652edb59 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:22.190750 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046900b-ca5b-4cfe-9c64-b054aa243cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:22.191595 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046900b-ca5b-4cfe-9c64-b054aa243cf6 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:22.192052 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046900b-ca5b-4cfe-9c64-b054aa243cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:22.192489 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046900b-ca5b-4cfe-9c64-b054aa243cf6 None admin] PATH_INFO: `/v3/services/network` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:22.193616 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9046900b-ca5b-4cfe-9c64-b054aa243cf6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:22.195235 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9046900b-ca5b-4cfe-9c64-b054aa243cf6 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:22.207213 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-9046900b-ca5b-4cfe-9c64-b054aa243cf6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 17 23:31:22.207213 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:22.207213 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 23:31:22.207213 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 23:31:22.207213 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 23:31:22.207213 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 23:31:22.207213 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 23:31:22.207213 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:22.207213 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 17 23:31:22.207213 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:22.207213 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 23:31:22.207213 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:22.207213 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:22.207213 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:22.207213 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:22.207213 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:22.207213 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:22.208365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:22.208365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:22.208365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:22.208365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:22.208365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:22.208365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:22.208365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 23:31:22.208365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 23:31:22.208365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 23:31:22.208365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 23:31:22.208365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:22.208365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:22.208365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:22.208365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:22.208365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:22.208365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:22.209199 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:22.209199 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:22.209199 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:22.209199 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:22.209199 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:22.209199 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:22.209199 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:22.209199 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:22.209199 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:22.209199 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:22.209199 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 23:31:22.209199 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:22.209199 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 17 23:31:22.209199 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:22.216236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 361/715] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:31:22 2022] GET /identity/v3/services/network => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:22.217366 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046900b-ca5b-4cfe-9c64-b054aa243cf6 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:22.229731 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:22.237852 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a3cff-5e49-4293-8610-617546136bb9 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:22.274175 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932bdcb-5090-403f-a4e1-d021b41d0bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:22.274789 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932bdcb-5090-403f-a4e1-d021b41d0bbb None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:22.275070 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932bdcb-5090-403f-a4e1-d021b41d0bbb None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:22.275345 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932bdcb-5090-403f-a4e1-d021b41d0bbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:22.306497 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:22.348199 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046900b-ca5b-4cfe-9c64-b054aa243cf6 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:22.351561 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932bdcb-5090-403f-a4e1-d021b41d0bbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:22.353064 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932bdcb-5090-403f-a4e1-d021b41d0bbb None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:22.383230 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e831f0-46ee-4864-bcde-a8801df74e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:22.384134 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e831f0-46ee-4864-bcde-a8801df74e33 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:22.384415 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e831f0-46ee-4864-bcde-a8801df74e33 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:22.384783 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e831f0-46ee-4864-bcde-a8801df74e33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:22.397017 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e831f0-46ee-4864-bcde-a8801df74e33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:22.398459 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e831f0-46ee-4864-bcde-a8801df74e33 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:22.412808 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-23e831f0-46ee-4864-bcde-a8801df74e33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:22.412808 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:22.412808 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:22.412808 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:22.412808 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:22.412808 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:22.412808 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:22.412808 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:22.412808 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:22.412808 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:22.412808 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:22.412808 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:22.412808 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:22.412808 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:22.412808 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:22.412808 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:22.413961 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:22.413961 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:22.413961 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:22.413961 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:22.413961 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:22.413961 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:22.413961 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:22.413961 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:22.413961 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:22.413961 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:22.413961 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:22.413961 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:22.413961 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:22.413961 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:22.413961 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:22.413961 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:22.413961 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:22.414859 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:22.414859 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:22.414859 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:22.414859 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 362/716] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:22.415095 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e831f0-46ee-4864-bcde-a8801df74e33 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:22.466378 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 355/717] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:31:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:31:22.467631 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a0b1d-c903-4c96-bd09-fa2cc5f12f6f None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:22.467913 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a0b1d-c903-4c96-bd09-fa2cc5f12f6f None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:22.468203 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a0b1d-c903-4c96-bd09-fa2cc5f12f6f None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:22.472025 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 356/718] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:22.472025 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a0b1d-c903-4c96-bd09-fa2cc5f12f6f None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:22.486856 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:22.532048 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e831f0-46ee-4864-bcde-a8801df74e33 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:22.537631 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:22.568481 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7a0e5-2973-4e38-893b-739be816c7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:22.569096 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7a0e5-2973-4e38-893b-739be816c7d8 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:22.569381 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7a0e5-2973-4e38-893b-739be816c7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:22.569659 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7a0e5-2973-4e38-893b-739be816c7d8 None admin] PATH_INFO: `/v3/services` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:22.570293 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f7a0e5-2973-4e38-893b-739be816c7d8 None admin] RBAC: Adding query filter params (name=network) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:22.570608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f7a0e5-2973-4e38-893b-739be816c7d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:22.572475 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f7a0e5-2973-4e38-893b-739be816c7d8 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:22.577379 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 363/719] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Aug 17 23:31:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:22.578163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7a0e5-2973-4e38-893b-739be816c7d8 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:22.589309 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a0b1d-c903-4c96-bd09-fa2cc5f12f6f None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:22.632297 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426555f-1485-4cb6-b13f-1050d971c049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:22.633315 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426555f-1485-4cb6-b13f-1050d971c049 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:22.633601 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426555f-1485-4cb6-b13f-1050d971c049 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:22.633877 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426555f-1485-4cb6-b13f-1050d971c049 None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:22.634525 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426555f-1485-4cb6-b13f-1050d971c049 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:22.634834 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426555f-1485-4cb6-b13f-1050d971c049 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:22.636801 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426555f-1485-4cb6-b13f-1050d971c049 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:22.652661 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 357/720] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:22.652661 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4e41e-17f5-4f2a-906c-ff68e8173ad2 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:22.652661 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4e41e-17f5-4f2a-906c-ff68e8173ad2 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:22.652661 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4e41e-17f5-4f2a-906c-ff68e8173ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:22.667821 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-55b4e41e-17f5-4f2a-906c-ff68e8173ad2 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:22.669263 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-55b4e41e-17f5-4f2a-906c-ff68e8173ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:22.689269 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7a0e5-2973-4e38-893b-739be816c7d8 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:22.714552 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:22.727943 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ab8319-a28f-4c61-a611-384e5b3a4663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:22.728642 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab8319-a28f-4c61-a611-384e5b3a4663 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:22.729396 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab8319-a28f-4c61-a611-384e5b3a4663 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:22.729686 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab8319-a28f-4c61-a611-384e5b3a4663 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:22.740272 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ab8319-a28f-4c61-a611-384e5b3a4663 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:22.741747 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ab8319-a28f-4c61-a611-384e5b3a4663 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:22.747681 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 358/721] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:31:22.748762 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4e41e-17f5-4f2a-906c-ff68e8173ad2 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:22.769707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-67ab8319-a28f-4c61-a611-384e5b3a4663 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:22.769707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:22.769707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:22.769707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:22.769707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:22.769707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:22.769707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:22.769707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:22.769707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:22.769707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:22.769707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:22.769707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:22.769707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:22.769707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:22.769707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:22.769707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:22.770703 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:22.770703 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:22.770703 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:22.770703 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:22.770703 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:22.770703 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:22.770703 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:22.770703 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:22.770703 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:22.770703 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:22.770703 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:22.770703 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:22.770703 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:22.770703 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:22.770703 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:22.770703 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:22.770703 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:22.771644 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:22.771644 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:22.771644 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:22.775554 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 364/722] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:22.776498 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ab8319-a28f-4c61-a611-384e5b3a4663 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:22.815656 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:22.879112 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4e41e-17f5-4f2a-906c-ff68e8173ad2 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:22.890222 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ab8319-a28f-4c61-a611-384e5b3a4663 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:22.916470 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a244a-6d8d-4d25-882f-85e520936a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:22.917184 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a244a-6d8d-4d25-882f-85e520936a8a None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:22.917466 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a244a-6d8d-4d25-882f-85e520936a8a None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:22.917737 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a244a-6d8d-4d25-882f-85e520936a8a None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:22.918420 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953a244a-6d8d-4d25-882f-85e520936a8a None admin] RBAC: Adding query filter params (type=network) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:22.918668 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953a244a-6d8d-4d25-882f-85e520936a8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:22.920260 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953a244a-6d8d-4d25-882f-85e520936a8a None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:22.924692 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eaeb0c-79d4-4939-b18d-cb4c4c62bcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:22.925322 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eaeb0c-79d4-4939-b18d-cb4c4c62bcdf None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:22.925605 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eaeb0c-79d4-4939-b18d-cb4c4c62bcdf None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:22.925881 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eaeb0c-79d4-4939-b18d-cb4c4c62bcdf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:22.928121 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 359/723] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Aug 17 23:31:22 2022] GET /identity/v3/services?type=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:22.928980 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a244a-6d8d-4d25-882f-85e520936a8a None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:22.944971 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eaeb0c-79d4-4939-b18d-cb4c4c62bcdf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:22.946600 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eaeb0c-79d4-4939-b18d-cb4c4c62bcdf None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:22.959916 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-31eaeb0c-79d4-4939-b18d-cb4c4c62bcdf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 17 23:31:22.959916 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:22.959916 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:22.959916 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:22.959916 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:22.959916 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:22.959916 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:22.959916 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:22.959916 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:22.959916 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:22.959916 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:22.959916 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:22.959916 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 23:31:22.959916 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 23:31:22.959916 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 23:31:22.959916 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 23:31:22.961006 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:22.961006 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:22.961006 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 23:31:22.961006 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:22.961006 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 23:31:22.961006 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:22.961006 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:22.961006 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:22.961006 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:22.961006 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:22.961006 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:22.961006 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:22.961006 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:22.961006 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:22.961006 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:22.961006 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:22.961006 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:22.961006 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:22.962001 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:22.962001 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:22.962001 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 23:31:22.962001 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 23:31:22.962001 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 23:31:22.962001 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 23:31:22.962001 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 23:31:22.962001 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 23:31:22.962001 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 17 23:31:22.962001 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:22.966673 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 365/724] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Aug 17 23:31:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:22.968103 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce20ba2-6d52-4f4f-8cb7-da979d54638e None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:22.968397 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce20ba2-6d52-4f4f-8cb7-da979d54638e None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:22.968674 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce20ba2-6d52-4f4f-8cb7-da979d54638e None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:22.970817 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 366/725] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:22.971109 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce20ba2-6d52-4f4f-8cb7-da979d54638e None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:23.012354 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:23.066474 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a244a-6d8d-4d25-882f-85e520936a8a None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:23.082476 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:23.095777 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce20ba2-6d52-4f4f-8cb7-da979d54638e None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:23.109421 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c243a759-3c3c-4345-8b28-e05d0152ced4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:23.110127 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243a759-3c3c-4345-8b28-e05d0152ced4 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:23.110409 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243a759-3c3c-4345-8b28-e05d0152ced4 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:23.110681 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243a759-3c3c-4345-8b28-e05d0152ced4 None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:23.111329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c243a759-3c3c-4345-8b28-e05d0152ced4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:23.111634 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c243a759-3c3c-4345-8b28-e05d0152ced4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:23.113059 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c243a759-3c3c-4345-8b28-e05d0152ced4 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:23.130208 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 360/726] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:23.130208 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972ad83-0c7a-4a4b-bf7a-e50d32f69317 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:23.130208 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972ad83-0c7a-4a4b-bf7a-e50d32f69317 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:23.130208 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972ad83-0c7a-4a4b-bf7a-e50d32f69317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:23.141335 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e350bdee-f6fe-4256-8689-e4da11e70b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:23.141953 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350bdee-f6fe-4256-8689-e4da11e70b10 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:23.142229 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350bdee-f6fe-4256-8689-e4da11e70b10 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:23.142499 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350bdee-f6fe-4256-8689-e4da11e70b10 None admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:23.143236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e350bdee-f6fe-4256-8689-e4da11e70b10 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:23.143544 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e350bdee-f6fe-4256-8689-e4da11e70b10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:23.145013 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e350bdee-f6fe-4256-8689-e4da11e70b10 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:23.146587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-3972ad83-0c7a-4a4b-bf7a-e50d32f69317 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:23.147756 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-3972ad83-0c7a-4a4b-bf7a-e50d32f69317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:23.163809 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 367/727] 10.222.0.26 () {56 vars in 1149 bytes} [Wed Aug 17 23:31:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:23.164942 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e350bdee-f6fe-4256-8689-e4da11e70b10 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:23.228463 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 361/728] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:31:23.228463 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3972ad83-0c7a-4a4b-bf7a-e50d32f69317 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:23.271074 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:23.281607 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e350bdee-f6fe-4256-8689-e4da11e70b10 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:23.323945 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dee7e8-a7f5-43a1-b022-46041d6591fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:23.323945 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dee7e8-a7f5-43a1-b022-46041d6591fa None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:23.323945 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dee7e8-a7f5-43a1-b022-46041d6591fa None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:23.323945 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dee7e8-a7f5-43a1-b022-46041d6591fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:23.335592 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dee7e8-a7f5-43a1-b022-46041d6591fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:31:23.343021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-e7dee7e8-a7f5-43a1-b022-46041d6591fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:31:23.343021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:23.343021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:23.343021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:23.343021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:23.343021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:23.343021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:23.343021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:23.343021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:23.343021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:23.343021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:23.343021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:23.343021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:31:23.343021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:31:23.343021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:31:23.343021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:31:23.344189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:31:23.344189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:31:23.344189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:31:23.344189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:31:23.344189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:23.344189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:23.344189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:23.344189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:23.344189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:23.344189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:23.344189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:23.344189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:23.344189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:23.344189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:23.344189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:23.344189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:23.345236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:23.345236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:23.345236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:23.345236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:23.345236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:31:23.345236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:31:23.345236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:31:23.345236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:31:23.345236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:31:23.345236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:31:23.345236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:31:23.345236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:23.345881 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 368/729] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 17 23:31:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:23.346433 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dee7e8-a7f5-43a1-b022-46041d6591fa None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:23.366017 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3972ad83-0c7a-4a4b-bf7a-e50d32f69317 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:23.378356 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:23.419346 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7dbe72-2df7-42ae-b76f-dc1fd4101020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:23.420492 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7dbe72-2df7-42ae-b76f-dc1fd4101020 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:23.421388 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7dbe72-2df7-42ae-b76f-dc1fd4101020 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:23.421953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7dbe72-2df7-42ae-b76f-dc1fd4101020 None admin] PATH_INFO: `/v3/projects/service` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:23.431288 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7dbe72-2df7-42ae-b76f-dc1fd4101020 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:23.433855 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7dbe72-2df7-42ae-b76f-dc1fd4101020 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:23.447213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-db7dbe72-2df7-42ae-b76f-dc1fd4101020 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:23.447213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:23.447213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:23.447213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:23.447213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:23.447213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:23.447213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:23.447213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:23.447213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:23.447213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:23.447213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:23.447213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:23.447213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 23:31:23.447213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 23:31:23.447213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 23:31:23.448378 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 23:31:23.448378 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:23.448378 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:23.448378 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:23.448378 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:23.448378 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:23.448378 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:23.448378 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:23.448378 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:23.448378 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:23.448378 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:23.448378 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:23.448378 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:23.448378 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:23.448378 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:23.448378 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:23.448378 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:23.449490 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 23:31:23.449490 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 23:31:23.449490 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 23:31:23.449490 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 23:31:23.449490 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 23:31:23.449490 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 23:31:23.449490 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:23.449490 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:23.453233 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 362/730] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:31:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:23.453233 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7dbe72-2df7-42ae-b76f-dc1fd4101020 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:23.486855 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dee7e8-a7f5-43a1-b022-46041d6591fa None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:23.530255 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f1a36a-346b-452f-84c1-18a7d6adcf58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:23.530869 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1a36a-346b-452f-84c1-18a7d6adcf58 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:23.531151 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1a36a-346b-452f-84c1-18a7d6adcf58 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:23.531426 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1a36a-346b-452f-84c1-18a7d6adcf58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:23.553773 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:23.560516 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7dbe72-2df7-42ae-b76f-dc1fd4101020 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:23.597792 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458bec82-fad6-470c-8982-d2f75efc78be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:23.598402 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458bec82-fad6-470c-8982-d2f75efc78be None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:23.598683 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458bec82-fad6-470c-8982-d2f75efc78be None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:23.598958 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458bec82-fad6-470c-8982-d2f75efc78be None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:23.599622 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458bec82-fad6-470c-8982-d2f75efc78be None admin] RBAC: Adding query filter params (name=admin) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:23.599927 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458bec82-fad6-470c-8982-d2f75efc78be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:23.601331 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458bec82-fad6-470c-8982-d2f75efc78be None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:23.606137 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f1a36a-346b-452f-84c1-18a7d6adcf58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:23.607914 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f1a36a-346b-452f-84c1-18a7d6adcf58 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:23.610064 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 363/731] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:31:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:23.610842 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458bec82-fad6-470c-8982-d2f75efc78be None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:23.637136 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:23.714703 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458bec82-fad6-470c-8982-d2f75efc78be None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:23.720338 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 369/732] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:31:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:31:23.720338 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f1a36a-346b-452f-84c1-18a7d6adcf58 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:23.755526 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09404545-58b4-4bdb-9f29-f1670bfa3014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:23.756368 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09404545-58b4-4bdb-9f29-f1670bfa3014 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:23.756672 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09404545-58b4-4bdb-9f29-f1670bfa3014 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:23.756944 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09404545-58b4-4bdb-9f29-f1670bfa3014 None admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:23.757633 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09404545-58b4-4bdb-9f29-f1670bfa3014 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:23.757940 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09404545-58b4-4bdb-9f29-f1670bfa3014 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:23.759376 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09404545-58b4-4bdb-9f29-f1670bfa3014 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:23.774849 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:23.779100 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 364/733] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 17 23:31:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:23.780043 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09404545-58b4-4bdb-9f29-f1670bfa3014 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:23.825145 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f1a36a-346b-452f-84c1-18a7d6adcf58 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:23.861523 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da4e2b-6f20-40b9-8e9c-247181f64081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:23.862128 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da4e2b-6f20-40b9-8e9c-247181f64081 None admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:23.862413 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da4e2b-6f20-40b9-8e9c-247181f64081 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:23.862688 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da4e2b-6f20-40b9-8e9c-247181f64081 None admin] PATH_INFO: `/v3/services` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:23.863334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27da4e2b-6f20-40b9-8e9c-247181f64081 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:23.864770 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27da4e2b-6f20-40b9-8e9c-247181f64081 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:23.874821 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:23.882610 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 370/734] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Aug 17 23:31:23 2022] POST /identity/v3/services => generated 251 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:31:23.883664 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da4e2b-6f20-40b9-8e9c-247181f64081 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:23.897127 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09404545-58b4-4bdb-9f29-f1670bfa3014 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:23.943589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372664d-bd47-4487-800d-f883d916b86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:23.944537 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372664d-bd47-4487-800d-f883d916b86d None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:23.944963 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372664d-bd47-4487-800d-f883d916b86d None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:23.945445 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372664d-bd47-4487-800d-f883d916b86d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:23.957090 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6372664d-bd47-4487-800d-f883d916b86d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:23.958853 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6372664d-bd47-4487-800d-f883d916b86d None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:23.975587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-6372664d-bd47-4487-800d-f883d916b86d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:23.975587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:23.975587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:23.975587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:23.975587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:23.975587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:23.975587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:23.975587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:23.975587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:23.975587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:23.975587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:23.975587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:23.975587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:23.975587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:23.975587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:23.975587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:23.976784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:23.976784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:23.976784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:23.976784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:23.976784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:23.976784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:23.976784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:23.976784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:23.976784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:23.976784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:23.976784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:23.976784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:23.976784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:23.976784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:23.976784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:23.976784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:23.976784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:23.977695 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:23.977695 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:23.977695 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:23.980485 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 365/735] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:23.981733 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372664d-bd47-4487-800d-f883d916b86d None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:24.037011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da4e2b-6f20-40b9-8e9c-247181f64081 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:24.057720 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:24.083258 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79d168-2db5-41ec-91f7-544b59dc5250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:24.084244 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79d168-2db5-41ec-91f7-544b59dc5250 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:24.084725 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79d168-2db5-41ec-91f7-544b59dc5250 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:24.085150 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79d168-2db5-41ec-91f7-544b59dc5250 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:24.098804 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f79d168-2db5-41ec-91f7-544b59dc5250 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:24.100400 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f79d168-2db5-41ec-91f7-544b59dc5250 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:24.110675 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372664d-bd47-4487-800d-f883d916b86d None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:24.115450 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-3f79d168-2db5-41ec-91f7-544b59dc5250 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:24.115450 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:24.115450 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:24.115450 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:24.115450 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:24.115450 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:24.115450 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:24.115450 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:24.115450 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:24.115450 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:24.115450 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:24.115450 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:24.115450 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:24.115450 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:24.115450 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:24.115450 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:24.116586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:24.116586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:24.116586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:24.116586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:24.116586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:24.116586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:24.116586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:24.116586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:24.116586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:24.116586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:24.116586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:24.116586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:24.116586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:24.116586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:24.116586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:24.116586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:24.116586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:24.122888 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:24.122888 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:24.122888 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:24.122888 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 371/736] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:24.122888 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79d168-2db5-41ec-91f7-544b59dc5250 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:24.159583 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:24.176004 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2b5d9-b66d-49c2-9e68-48d2d35bd443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:24.176004 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2b5d9-b66d-49c2-9e68-48d2d35bd443 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:24.176004 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2b5d9-b66d-49c2-9e68-48d2d35bd443 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:24.176004 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2b5d9-b66d-49c2-9e68-48d2d35bd443 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:24.176479 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2b5d9-b66d-49c2-9e68-48d2d35bd443 None admin] RBAC: Adding query filter params (role.id=33bd60d3b0e94dcbbfe97e87f91a8d78, scope.project.id=4de86915ef4340248ba8a8f51d56594e, user.id=ec1dcbe43b0749c3bcc4af9100d771d1) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:24.176479 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2b5d9-b66d-49c2-9e68-48d2d35bd443 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:24.177524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2b5d9-b66d-49c2-9e68-48d2d35bd443 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:24.188045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 366/737] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:31:23 2022] GET /identity/v3/role_assignments?user.id=ec1dcbe43b0749c3bcc4af9100d771d1&scope.project.id=4de86915ef4340248ba8a8f51d56594e&role.id=33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:24.188990 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2b5d9-b66d-49c2-9e68-48d2d35bd443 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:24.296073 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79d168-2db5-41ec-91f7-544b59dc5250 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:24.356919 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3c70c1-0e38-4e99-8687-249b9754101e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:24.357548 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c70c1-0e38-4e99-8687-249b9754101e None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:24.357830 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c70c1-0e38-4e99-8687-249b9754101e None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:24.358103 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c70c1-0e38-4e99-8687-249b9754101e None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:24.358762 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3c70c1-0e38-4e99-8687-249b9754101e None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:24.359065 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3c70c1-0e38-4e99-8687-249b9754101e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:24.360655 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3c70c1-0e38-4e99-8687-249b9754101e None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:24.376131 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 372/738] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:24.381301 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3c70c1-0e38-4e99-8687-249b9754101e None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:24.383282 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2b5d9-b66d-49c2-9e68-48d2d35bd443 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:24.412461 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:24.424019 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c27ed-48a1-4d6c-a773-6daa81c57b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:24.424600 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c27ed-48a1-4d6c-a773-6daa81c57b19 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:24.424880 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c27ed-48a1-4d6c-a773-6daa81c57b19 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:24.425154 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c27ed-48a1-4d6c-a773-6daa81c57b19 None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:24.425968 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c27ed-48a1-4d6c-a773-6daa81c57b19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:24.426277 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c27ed-48a1-4d6c-a773-6daa81c57b19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:24.427780 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c27ed-48a1-4d6c-a773-6daa81c57b19 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:24.442244 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 367/739] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:24.455315 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c27ed-48a1-4d6c-a773-6daa81c57b19 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:24.523342 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:24.539421 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3c70c1-0e38-4e99-8687-249b9754101e None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:24.573143 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c27ed-48a1-4d6c-a773-6daa81c57b19 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:24.601943 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda13f3f-b33f-4080-b5ca-0787ecec7c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:24.602716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda13f3f-b33f-4080-b5ca-0787ecec7c18 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:24.602909 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda13f3f-b33f-4080-b5ca-0787ecec7c18 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:24.603136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda13f3f-b33f-4080-b5ca-0787ecec7c18 None admin] PATH_INFO: `/v3/users/swift` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:24.615337 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda13f3f-b33f-4080-b5ca-0787ecec7c18 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:24.617729 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a68807-53b1-4246-b77f-dcdaf628d70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:24.618648 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a68807-53b1-4246-b77f-dcdaf628d70d None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:24.619146 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a68807-53b1-4246-b77f-dcdaf628d70d None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:24.620189 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a68807-53b1-4246-b77f-dcdaf628d70d None admin] PATH_INFO: `/v3/projects/service` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:24.621976 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda13f3f-b33f-4080-b5ca-0787ecec7c18 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:24.631781 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a68807-53b1-4246-b77f-dcdaf628d70d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:24.638052 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a68807-53b1-4246-b77f-dcdaf628d70d None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:24.644197 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-dda13f3f-b33f-4080-b5ca-0787ecec7c18 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 17 23:31:24.644197 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:24.644197 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:24.644197 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:24.644197 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:24.644197 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:24.644197 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:24.644197 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:24.644197 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:24.644197 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:24.644197 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:24.644197 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:24.644197 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 23:31:24.644197 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 23:31:24.644197 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 23:31:24.644197 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 23:31:24.645189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:24.645189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:24.645189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 23:31:24.645189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:24.645189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 23:31:24.645189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:24.645189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:24.645189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:24.645189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:24.645189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:24.645189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:24.645189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:24.645189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:24.645189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:24.645189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:24.645189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:24.645189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:24.645189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:24.646234 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:24.646234 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:24.646234 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 23:31:24.646234 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 23:31:24.646234 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 23:31:24.646234 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 23:31:24.646234 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 23:31:24.646234 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 23:31:24.646234 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 17 23:31:24.646234 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:24.649133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 373/740] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 17 23:31:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:24.658730 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-52a68807-53b1-4246-b77f-dcdaf628d70d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:24.658730 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:24.658730 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:24.658730 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:24.658730 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:24.658730 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:24.658730 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:24.658730 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:24.658730 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:24.658730 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:24.658730 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:24.658730 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:24.658730 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 23:31:24.658730 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 23:31:24.658730 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 23:31:24.659646 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 23:31:24.659646 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:24.659646 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:24.659646 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:24.659646 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:24.659646 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:24.659646 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:24.659646 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:24.659646 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:24.659646 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:24.659646 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:24.659646 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:24.659646 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:24.659646 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:24.659646 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:24.659646 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:24.659646 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:24.660596 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 23:31:24.660596 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 23:31:24.660596 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 23:31:24.660596 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 23:31:24.660596 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 23:31:24.660596 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 23:31:24.660596 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:24.660596 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:24.660596 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 368/741] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:31:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:31:24.661716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda13f3f-b33f-4080-b5ca-0787ecec7c18 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:24.665216 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a68807-53b1-4246-b77f-dcdaf628d70d None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:24.789827 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a68807-53b1-4246-b77f-dcdaf628d70d None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:24.808626 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:24.826029 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f699bb-6149-43c8-9217-dbf537038d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:24.826523 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f699bb-6149-43c8-9217-dbf537038d3c None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:24.827064 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f699bb-6149-43c8-9217-dbf537038d3c None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:24.827638 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f699bb-6149-43c8-9217-dbf537038d3c None admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:24.828579 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f699bb-6149-43c8-9217-dbf537038d3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:24.829092 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f699bb-6149-43c8-9217-dbf537038d3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:24.830914 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f699bb-6149-43c8-9217-dbf537038d3c None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:24.844214 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 369/742] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 17 23:31:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:24.856677 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f699bb-6149-43c8-9217-dbf537038d3c None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:24.881425 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:24.906731 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda13f3f-b33f-4080-b5ca-0787ecec7c18 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:24.942887 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d068d5a7-8375-4e3c-83e4-f8aeb2ee7042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:24.943643 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068d5a7-8375-4e3c-83e4-f8aeb2ee7042 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:24.943935 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068d5a7-8375-4e3c-83e4-f8aeb2ee7042 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:24.944236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068d5a7-8375-4e3c-83e4-f8aeb2ee7042 None admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:24.944970 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d068d5a7-8375-4e3c-83e4-f8aeb2ee7042 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:24.945279 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d068d5a7-8375-4e3c-83e4-f8aeb2ee7042 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:24.946941 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d068d5a7-8375-4e3c-83e4-f8aeb2ee7042 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:24.965560 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f699bb-6149-43c8-9217-dbf537038d3c None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:24.966487 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 374/743] 10.222.0.26 () {56 vars in 1135 bytes} [Wed Aug 17 23:31:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:31:24.972238 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190fc8e-e6d3-4c91-a608-77b9e1a556ff None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:24.972530 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190fc8e-e6d3-4c91-a608-77b9e1a556ff None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:24.972810 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190fc8e-e6d3-4c91-a608-77b9e1a556ff None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:24.974435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 375/744] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:24.975157 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4190fc8e-e6d3-4c91-a608-77b9e1a556ff None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:25.006822 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5178dbdc-c377-4685-8707-43844ad905be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:25.007433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5178dbdc-c377-4685-8707-43844ad905be None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:25.007713 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5178dbdc-c377-4685-8707-43844ad905be None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:25.008003 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5178dbdc-c377-4685-8707-43844ad905be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:25.008835 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5178dbdc-c377-4685-8707-43844ad905be None admin] RBAC: Adding query filter params (role.id=b64f80e6f3ea4ad49b94acdde42ccd87, scope.project.id=4de86915ef4340248ba8a8f51d56594e, user.id=d0a13dbffb4a49508b7fb661cf84e803) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:25.009147 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5178dbdc-c377-4685-8707-43844ad905be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:25.010731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5178dbdc-c377-4685-8707-43844ad905be None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:25.016891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 370/745] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:31:24 2022] GET /identity/v3/role_assignments?user.id=d0a13dbffb4a49508b7fb661cf84e803&scope.project.id=4de86915ef4340248ba8a8f51d56594e&role.id=b64f80e6f3ea4ad49b94acdde42ccd87 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:25.018121 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb692cf-273d-437a-b514-a2bed32ef497 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:25.018404 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb692cf-273d-437a-b514-a2bed32ef497 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:25.018681 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb692cf-273d-437a-b514-a2bed32ef497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:25.038786 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-bfb692cf-273d-437a-b514-a2bed32ef497 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:25.040134 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-bfb692cf-273d-437a-b514-a2bed32ef497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:25.087982 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:25.090945 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4190fc8e-e6d3-4c91-a608-77b9e1a556ff None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:25.132091 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 371/746] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:25 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 23:31:25.151933 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99babcfe-917b-4671-8037-a14742c60200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:25.152637 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99babcfe-917b-4671-8037-a14742c60200 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:25.153262 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99babcfe-917b-4671-8037-a14742c60200 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:25.153536 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99babcfe-917b-4671-8037-a14742c60200 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:25.176640 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:25.247315 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99babcfe-917b-4671-8037-a14742c60200 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:25.248816 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99babcfe-917b-4671-8037-a14742c60200 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:25.355379 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 376/747] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:31:24 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 23:31:25.360683 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb692cf-273d-437a-b514-a2bed32ef497 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:25.483186 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb692cf-273d-437a-b514-a2bed32ef497 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:25.513742 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99babcfe-917b-4671-8037-a14742c60200 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:25.535750 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a30e5-6e0c-4b81-b80f-3c7704d9d547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:25.536416 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a30e5-6e0c-4b81-b80f-3c7704d9d547 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:25.536697 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a30e5-6e0c-4b81-b80f-3c7704d9d547 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:25.536971 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a30e5-6e0c-4b81-b80f-3c7704d9d547 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:25.550831 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60a30e5-6e0c-4b81-b80f-3c7704d9d547 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:25.552478 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60a30e5-6e0c-4b81-b80f-3c7704d9d547 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:25.568544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-e60a30e5-6e0c-4b81-b80f-3c7704d9d547 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:25.568544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:25.568544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:25.568544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:25.568544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:25.568544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:25.568544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:25.568544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:25.568544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:25.568544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:25.568544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:25.568544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:25.568544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:25.568544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:25.568544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:25.568544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:25.569501 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:25.569501 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:25.569501 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:25.569501 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:25.569501 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:25.569501 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:25.569501 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:25.569501 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:25.569501 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:25.569501 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:25.569501 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:25.569501 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:25.569501 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:25.569501 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:25.569501 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:25.569501 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:25.569501 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:25.570401 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:25.570401 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:25.570401 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:25.570401 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 372/748] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:25.575089 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a30e5-6e0c-4b81-b80f-3c7704d9d547 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:25.603558 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:25.620852 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:25.722206 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99babcfe-917b-4671-8037-a14742c60200 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:25.726217 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a30e5-6e0c-4b81-b80f-3c7704d9d547 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:25.772487 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bad58ba-940d-4232-8cd2-0499d86501b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:25.773109 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad58ba-940d-4232-8cd2-0499d86501b5 None admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:25.773393 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad58ba-940d-4232-8cd2-0499d86501b5 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:25.773677 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad58ba-940d-4232-8cd2-0499d86501b5 None admin] PATH_INFO: `/v3/services` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:25.774329 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bad58ba-940d-4232-8cd2-0499d86501b5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:25.776296 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bad58ba-940d-4232-8cd2-0499d86501b5 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:25.790878 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb68c41-ac4d-49af-b1c7-b95b653916cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:25.791658 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb68c41-ac4d-49af-b1c7-b95b653916cc None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:25.792067 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb68c41-ac4d-49af-b1c7-b95b653916cc None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:25.792466 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb68c41-ac4d-49af-b1c7-b95b653916cc None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:25.794838 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb68c41-ac4d-49af-b1c7-b95b653916cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:25.796005 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb68c41-ac4d-49af-b1c7-b95b653916cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:25.798824 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb68c41-ac4d-49af-b1c7-b95b653916cc None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:25.803732 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 377/749] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Aug 17 23:31:25 2022] POST /identity/v3/services => generated 240 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:31:25.816209 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 373/750] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:25.825224 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bad58ba-940d-4232-8cd2-0499d86501b5 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:25.976869 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:25.993968 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bad58ba-940d-4232-8cd2-0499d86501b5 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:26.043035 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847732c7-b332-4ff2-bc46-e42c4afe49ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:26.043684 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847732c7-b332-4ff2-bc46-e42c4afe49ea None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:26.043999 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847732c7-b332-4ff2-bc46-e42c4afe49ea None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:26.044270 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847732c7-b332-4ff2-bc46-e42c4afe49ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:26.057415 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847732c7-b332-4ff2-bc46-e42c4afe49ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:26.058890 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847732c7-b332-4ff2-bc46-e42c4afe49ea None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:26.074760 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-847732c7-b332-4ff2-bc46-e42c4afe49ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:26.074760 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:26.074760 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:26.074760 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:26.074760 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:26.074760 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:26.074760 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:26.074760 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:26.074760 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:26.074760 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:26.074760 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:26.074760 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:26.074760 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 23:31:26.074760 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 23:31:26.074760 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 23:31:26.075709 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 23:31:26.075709 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:26.075709 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:26.075709 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:26.075709 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:26.075709 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:26.075709 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:26.075709 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:26.075709 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:26.075709 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:26.075709 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:26.075709 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:26.075709 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:26.075709 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:26.075709 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:26.075709 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:26.075709 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:26.076662 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 23:31:26.076662 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 23:31:26.076662 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 23:31:26.076662 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 23:31:26.076662 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 23:31:26.076662 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 23:31:26.076662 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:26.076662 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:26.076662 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 378/751] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:31:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:26.083297 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb68c41-ac4d-49af-b1c7-b95b653916cc None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:26.197304 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:26.222800 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb68c41-ac4d-49af-b1c7-b95b653916cc None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:26.269527 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83dfcd5-18c4-4198-a0b0-0af6378ce5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:26.270450 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83dfcd5-18c4-4198-a0b0-0af6378ce5c6 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:26.271103 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83dfcd5-18c4-4198-a0b0-0af6378ce5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:26.271616 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83dfcd5-18c4-4198-a0b0-0af6378ce5c6 None admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:26.272540 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83dfcd5-18c4-4198-a0b0-0af6378ce5c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:26.273038 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83dfcd5-18c4-4198-a0b0-0af6378ce5c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:26.274662 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83dfcd5-18c4-4198-a0b0-0af6378ce5c6 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:26.292720 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 374/752] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 17 23:31:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:26.301061 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847732c7-b332-4ff2-bc46-e42c4afe49ea None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:26.340665 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdafe62-ff1a-4113-aa65-e5281b099c02 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:26.341289 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdafe62-ff1a-4113-aa65-e5281b099c02 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:26.341820 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdafe62-ff1a-4113-aa65-e5281b099c02 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:26.343614 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 375/753] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 23:31:26.349161 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae241a9-e1e6-455e-81ca-339809e739eb None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:26.349723 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae241a9-e1e6-455e-81ca-339809e739eb None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:26.350267 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae241a9-e1e6-455e-81ca-339809e739eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:26.379772 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-8ae241a9-e1e6-455e-81ca-339809e739eb None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:26.382062 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-8ae241a9-e1e6-455e-81ca-339809e739eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:26.442899 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847732c7-b332-4ff2-bc46-e42c4afe49ea None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:26.482829 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 376/754] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 23:31:26.484912 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e8874-b30f-474d-8b05-387320e19282 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:26.485412 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e8874-b30f-474d-8b05-387320e19282 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:26.485893 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e8874-b30f-474d-8b05-387320e19282 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:26.487576 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 377/755] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:26.496006 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c4e6e-f62d-4e42-8eb2-690c37fae2f1 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:26.496006 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c4e6e-f62d-4e42-8eb2-690c37fae2f1 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:26.496006 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c4e6e-f62d-4e42-8eb2-690c37fae2f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:26.508660 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5291672-ecf0-44eb-a31d-f7f5e151f302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:26.509573 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5291672-ecf0-44eb-a31d-f7f5e151f302 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:26.510659 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5291672-ecf0-44eb-a31d-f7f5e151f302 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:26.511134 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5291672-ecf0-44eb-a31d-f7f5e151f302 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:26.512193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5291672-ecf0-44eb-a31d-f7f5e151f302 None admin] RBAC: Adding query filter params (role.id=33bd60d3b0e94dcbbfe97e87f91a8d78, scope.project.id=4de86915ef4340248ba8a8f51d56594e, user.id=ee422ea90557444fabdc9c292865b9b8) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:26.512381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-a55c4e6e-f62d-4e42-8eb2-690c37fae2f1 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:26.512882 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5291672-ecf0-44eb-a31d-f7f5e151f302 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:26.513598 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-a55c4e6e-f62d-4e42-8eb2-690c37fae2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:26.514740 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5291672-ecf0-44eb-a31d-f7f5e151f302 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:26.537973 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 379/756] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:31:26 2022] GET /identity/v3/role_assignments?user.id=ee422ea90557444fabdc9c292865b9b8&scope.project.id=4de86915ef4340248ba8a8f51d56594e&role.id=33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:26.587575 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 378/757] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:31:26.813961 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5291672-ecf0-44eb-a31d-f7f5e151f302 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:26.840660 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:26.878616 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c4e6e-f62d-4e42-8eb2-690c37fae2f1 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:26.883986 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:26.933208 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5291672-ecf0-44eb-a31d-f7f5e151f302 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:27.000819 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e139f5d5-2767-4bf2-8df5-cc88c445e742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:27.003553 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139f5d5-2767-4bf2-8df5-cc88c445e742 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:27.005287 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139f5d5-2767-4bf2-8df5-cc88c445e742 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:27.005827 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139f5d5-2767-4bf2-8df5-cc88c445e742 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:27.006625 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e139f5d5-2767-4bf2-8df5-cc88c445e742 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:27.009148 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e139f5d5-2767-4bf2-8df5-cc88c445e742 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:27.030256 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-e139f5d5-2767-4bf2-8df5-cc88c445e742 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 17 23:31:27.030256 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:27.030256 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 23:31:27.030256 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 23:31:27.030256 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 23:31:27.030256 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 23:31:27.030256 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 23:31:27.030256 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:27.030256 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 17 23:31:27.030256 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:27.030256 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 23:31:27.030256 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:27.030256 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:27.030256 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:27.030256 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:27.030256 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:27.030256 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:27.031357 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:27.031357 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:27.031357 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:27.031357 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:27.031357 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:27.031357 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:27.031357 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 23:31:27.031357 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 23:31:27.031357 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 23:31:27.031357 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 23:31:27.031357 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:27.031357 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:27.031357 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:27.031357 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:27.031357 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:27.031357 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:27.032264 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:27.032264 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:27.032264 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:27.032264 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:27.032264 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:27.032264 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:27.032264 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:27.032264 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:27.032264 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:27.032264 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:27.032264 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 23:31:27.032264 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:27.032264 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 17 23:31:27.032264 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:27.037210 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 380/758] 10.222.0.26 () {56 vars in 1129 bytes} [Wed Aug 17 23:31:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:27.043057 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e139f5d5-2767-4bf2-8df5-cc88c445e742 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:27.090829 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c4e6e-f62d-4e42-8eb2-690c37fae2f1 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:27.165294 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef84661c-c987-4601-a616-73f4b36b05a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:27.166918 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84661c-c987-4601-a616-73f4b36b05a7 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:27.167454 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84661c-c987-4601-a616-73f4b36b05a7 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:27.167929 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84661c-c987-4601-a616-73f4b36b05a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:27.182724 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:27.189119 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef84661c-c987-4601-a616-73f4b36b05a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:31:27.194718 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-ef84661c-c987-4601-a616-73f4b36b05a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:31:27.194718 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:27.194718 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:27.194718 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:27.194718 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:27.194718 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:27.194718 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:27.194718 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:27.194718 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:27.194718 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:27.194718 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:27.194718 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:27.194718 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:31:27.194718 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:31:27.194718 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:31:27.194718 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:31:27.195777 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:31:27.195777 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:31:27.195777 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:31:27.195777 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:31:27.195777 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:27.195777 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:27.195777 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:27.195777 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:27.195777 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:27.195777 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:27.195777 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:27.195777 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:27.195777 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:27.195777 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:27.195777 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:27.195777 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:27.196685 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:27.196685 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:27.196685 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:27.196685 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:27.196685 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:31:27.196685 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:31:27.196685 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:31:27.196685 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:31:27.196685 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:31:27.196685 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:31:27.196685 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:31:27.196685 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:27.199817 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 379/759] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 17 23:31:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:31:27.214664 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e139f5d5-2767-4bf2-8df5-cc88c445e742 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:27.217469 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef84661c-c987-4601-a616-73f4b36b05a7 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:27.288341 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:27.288649 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d44550-09d1-498a-ae22-7a50a34acaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:27.289278 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d44550-09d1-498a-ae22-7a50a34acaf8 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:27.289529 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d44550-09d1-498a-ae22-7a50a34acaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:27.289802 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d44550-09d1-498a-ae22-7a50a34acaf8 None admin] PATH_INFO: `/v3/services` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:27.290439 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d44550-09d1-498a-ae22-7a50a34acaf8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:27.290744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d44550-09d1-498a-ae22-7a50a34acaf8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:27.292192 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d44550-09d1-498a-ae22-7a50a34acaf8 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:27.306292 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 381/760] 10.222.0.26 () {56 vars in 1124 bytes} [Wed Aug 17 23:31:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:27.311420 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d44550-09d1-498a-ae22-7a50a34acaf8 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:27.334739 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef84661c-c987-4601-a616-73f4b36b05a7 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:27.395893 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8636b-f654-4fb4-879b-61f46c6291e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:27.396848 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8636b-f654-4fb4-879b-61f46c6291e7 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:27.397352 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8636b-f654-4fb4-879b-61f46c6291e7 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:27.397848 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8636b-f654-4fb4-879b-61f46c6291e7 None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:27.398715 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8636b-f654-4fb4-879b-61f46c6291e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:27.399217 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8636b-f654-4fb4-879b-61f46c6291e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:27.401492 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8636b-f654-4fb4-879b-61f46c6291e7 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:27.415559 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 380/761] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:31:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:31:27.422605 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8636b-f654-4fb4-879b-61f46c6291e7 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:27.459105 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d44550-09d1-498a-ae22-7a50a34acaf8 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:27.521664 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd4180c-1c48-4a9d-9433-9182e15bf8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:27.522707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd4180c-1c48-4a9d-9433-9182e15bf8e4 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:27.523233 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd4180c-1c48-4a9d-9433-9182e15bf8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:27.523716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd4180c-1c48-4a9d-9433-9182e15bf8e4 None admin] PATH_INFO: `/v3/services` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:27.524666 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd4180c-1c48-4a9d-9433-9182e15bf8e4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:27.525187 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd4180c-1c48-4a9d-9433-9182e15bf8e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:27.527409 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd4180c-1c48-4a9d-9433-9182e15bf8e4 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:27.535301 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 382/762] 10.222.0.26 () {56 vars in 1124 bytes} [Wed Aug 17 23:31:27 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:31:27.537000 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8688ce70-e736-4eda-9c97-d8809b69518b None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:27.537500 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8688ce70-e736-4eda-9c97-d8809b69518b None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:27.537994 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8688ce70-e736-4eda-9c97-d8809b69518b None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:27.540009 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 383/763] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:27.545539 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8688ce70-e736-4eda-9c97-d8809b69518b None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:27.564315 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:27.661479 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8688ce70-e736-4eda-9c97-d8809b69518b None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:27.698279 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce2c445-6142-44f3-8e12-82c903354299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:27.698897 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2c445-6142-44f3-8e12-82c903354299 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:27.699177 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2c445-6142-44f3-8e12-82c903354299 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:27.699551 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2c445-6142-44f3-8e12-82c903354299 None admin] PATH_INFO: `/v3/endpoints` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:27.700342 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce2c445-6142-44f3-8e12-82c903354299 None admin] RBAC: Adding query filter params (interface=public, service_id=e068e130fcaa48eaabb63424af28566a, region_id=RegionOne) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:27.700783 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce2c445-6142-44f3-8e12-82c903354299 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:27.702167 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce2c445-6142-44f3-8e12-82c903354299 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:27.712611 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 384/764] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Aug 17 23:31:27 2022] GET /identity/v3/endpoints?service_id=e068e130fcaa48eaabb63424af28566a&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:27.714590 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee620b32-e27c-488c-bd88-c0868224edb5 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:27.714878 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee620b32-e27c-488c-bd88-c0868224edb5 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:27.715257 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee620b32-e27c-488c-bd88-c0868224edb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:27.721156 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8636b-f654-4fb4-879b-61f46c6291e7 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:27.736548 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-ee620b32-e27c-488c-bd88-c0868224edb5 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:27.737918 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-ee620b32-e27c-488c-bd88-c0868224edb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:27.748789 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:27.762432 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caec92b-9bd9-457e-9b36-e7f7549c244c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:27.763307 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caec92b-9bd9-457e-9b36-e7f7549c244c None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:27.763732 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caec92b-9bd9-457e-9b36-e7f7549c244c None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:27.764206 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caec92b-9bd9-457e-9b36-e7f7549c244c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:27.776527 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caec92b-9bd9-457e-9b36-e7f7549c244c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:27.778602 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caec92b-9bd9-457e-9b36-e7f7549c244c None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:27.797773 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-3caec92b-9bd9-457e-9b36-e7f7549c244c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:27.797773 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:27.797773 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:27.797773 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:27.797773 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:27.797773 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:27.797773 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:27.797773 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:27.797773 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:27.797773 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:27.797773 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:27.797773 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:27.797773 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:27.797773 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:27.797773 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:27.797773 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:27.798978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:27.798978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:27.798978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:27.798978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:27.798978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:27.798978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:27.798978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:27.798978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:27.798978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:27.798978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:27.798978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:27.798978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:27.798978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:27.798978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:27.798978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:27.798978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:27.798978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:27.800062 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:27.800062 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:27.800062 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:27.801808 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 381/765] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:27.802742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caec92b-9bd9-457e-9b36-e7f7549c244c None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:27.819404 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 385/766] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:31:27.824192 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee620b32-e27c-488c-bd88-c0868224edb5 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:27.846215 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:27.983667 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caec92b-9bd9-457e-9b36-e7f7549c244c None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:28.000368 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee620b32-e27c-488c-bd88-c0868224edb5 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:28.049000 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca876f7-2de2-4f7a-ae77-2293d369451b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:28.049603 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca876f7-2de2-4f7a-ae77-2293d369451b None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:28.049884 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca876f7-2de2-4f7a-ae77-2293d369451b None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:28.050154 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca876f7-2de2-4f7a-ae77-2293d369451b None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:28.050759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca876f7-2de2-4f7a-ae77-2293d369451b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:28.052152 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca876f7-2de2-4f7a-ae77-2293d369451b None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:28.058445 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 382/767] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Aug 17 23:31:27 2022] GET /identity/v3/services => generated 772 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:28.073392 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef7ea5-d3f4-4723-96d0-1560a6d2cef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:28.074098 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef7ea5-d3f4-4723-96d0-1560a6d2cef6 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:28.074310 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef7ea5-d3f4-4723-96d0-1560a6d2cef6 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:28.074583 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef7ea5-d3f4-4723-96d0-1560a6d2cef6 None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:28.075245 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ef7ea5-d3f4-4723-96d0-1560a6d2cef6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:28.075546 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ef7ea5-d3f4-4723-96d0-1560a6d2cef6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:28.076968 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ef7ea5-d3f4-4723-96d0-1560a6d2cef6 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:28.094562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 386/768] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:28.100200 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca876f7-2de2-4f7a-ae77-2293d369451b None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:28.146981 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef7ea5-d3f4-4723-96d0-1560a6d2cef6 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:28.153362 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:28.223854 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca876f7-2de2-4f7a-ae77-2293d369451b None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:28.263865 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c2cd94-80ba-4bcd-98f9-e3759e50714f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:28.264461 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2cd94-80ba-4bcd-98f9-e3759e50714f None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:28.264739 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2cd94-80ba-4bcd-98f9-e3759e50714f None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:28.265005 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2cd94-80ba-4bcd-98f9-e3759e50714f None admin] PATH_INFO: `/v3/users/nova` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:28.273638 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c2cd94-80ba-4bcd-98f9-e3759e50714f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:28.275133 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c2cd94-80ba-4bcd-98f9-e3759e50714f None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:28.288880 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-f7c2cd94-80ba-4bcd-98f9-e3759e50714f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 17 23:31:28.288880 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:28.288880 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:28.288880 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:28.288880 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:28.288880 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:28.288880 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:28.288880 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:28.288880 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:28.288880 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:28.288880 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:28.288880 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:28.288880 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 23:31:28.288880 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 23:31:28.288880 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 23:31:28.288880 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 23:31:28.289921 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:28.289921 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:28.289921 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 23:31:28.289921 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:28.289921 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 23:31:28.289921 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:28.289921 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:28.289921 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:28.289921 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:28.289921 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:28.289921 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:28.289921 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:28.289921 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:28.289921 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:28.289921 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:28.289921 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:28.289921 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:28.289921 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:28.290861 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:28.290861 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:28.290861 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 23:31:28.290861 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 23:31:28.290861 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 23:31:28.290861 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 23:31:28.290861 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 23:31:28.290861 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 23:31:28.290861 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 17 23:31:28.290861 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:28.290861 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 383/769] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Aug 17 23:31:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:28.291582 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105e91e-e6be-4bc5-ba07-638a96d3f95d None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:28.292240 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105e91e-e6be-4bc5-ba07-638a96d3f95d None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:28.292240 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105e91e-e6be-4bc5-ba07-638a96d3f95d None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:28.293572 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 384/770] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:28.295693 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e105e91e-e6be-4bc5-ba07-638a96d3f95d None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:28.357036 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:28.402363 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef7ea5-d3f4-4723-96d0-1560a6d2cef6 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:28.411089 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:28.417002 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e105e91e-e6be-4bc5-ba07-638a96d3f95d None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:28.450300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d73f8e4-cbaf-44ef-98f8-882a271862b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:28.450300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73f8e4-cbaf-44ef-98f8-882a271862b9 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:28.450300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73f8e4-cbaf-44ef-98f8-882a271862b9 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:28.450300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73f8e4-cbaf-44ef-98f8-882a271862b9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:28.463902 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d73f8e4-cbaf-44ef-98f8-882a271862b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:31:28.465967 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60429239-7c19-4115-8a3e-b137e0e48239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:28.466875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60429239-7c19-4115-8a3e-b137e0e48239 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:28.467426 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60429239-7c19-4115-8a3e-b137e0e48239 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:28.467910 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60429239-7c19-4115-8a3e-b137e0e48239 None admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:28.468923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-3d73f8e4-cbaf-44ef-98f8-882a271862b9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:31:28.468923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:28.468923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:28.468923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:28.468923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:28.468923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:28.468923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:28.468923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:28.468923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:28.468923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:28.468923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:28.468923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:28.468923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:31:28.468923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:31:28.468923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:31:28.468923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:31:28.469892 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:31:28.469892 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:31:28.469892 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:31:28.469892 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:31:28.469892 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:28.469892 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:28.469892 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:28.469892 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:28.469892 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:28.469892 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:28.469892 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:28.469892 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:28.469892 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:28.469892 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:28.469892 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:28.469892 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:28.470760 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:28.470760 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:28.470760 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:28.470760 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:28.470760 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:31:28.470760 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:31:28.470760 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:31:28.470760 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:31:28.470760 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:31:28.470760 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:31:28.470760 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:31:28.470760 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:28.470760 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 387/771] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 17 23:31:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:31:28.476533 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb395ea5-37df-42b1-97ad-02211b085b98 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:28.476533 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb395ea5-37df-42b1-97ad-02211b085b98 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:28.476533 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb395ea5-37df-42b1-97ad-02211b085b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:28.476938 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60429239-7c19-4115-8a3e-b137e0e48239 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:28.477542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60429239-7c19-4115-8a3e-b137e0e48239 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:28.479377 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60429239-7c19-4115-8a3e-b137e0e48239 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:28.492701 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-cb395ea5-37df-42b1-97ad-02211b085b98 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:28.493896 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-cb395ea5-37df-42b1-97ad-02211b085b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:28.500191 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 385/772] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Aug 17 23:31:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:31:28.501348 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60429239-7c19-4115-8a3e-b137e0e48239 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:28.591445 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 388/773] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:31:28.592599 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb395ea5-37df-42b1-97ad-02211b085b98 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:28.610919 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60429239-7c19-4115-8a3e-b137e0e48239 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:28.650513 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f3ee0e-ce6b-4b3a-9da0-da14ea785ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:28.651680 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3ee0e-ce6b-4b3a-9da0-da14ea785ec1 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:28.652204 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3ee0e-ce6b-4b3a-9da0-da14ea785ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:28.652973 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3ee0e-ce6b-4b3a-9da0-da14ea785ec1 None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:28.653845 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f3ee0e-ce6b-4b3a-9da0-da14ea785ec1 None admin] RBAC: Adding query filter params (name=service) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:28.654409 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f3ee0e-ce6b-4b3a-9da0-da14ea785ec1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:28.656025 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f3ee0e-ce6b-4b3a-9da0-da14ea785ec1 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:28.665732 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 386/774] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 17 23:31:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:28.670321 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:28.671533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f3ee0e-ce6b-4b3a-9da0-da14ea785ec1 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:28.696476 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:28.721305 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb395ea5-37df-42b1-97ad-02211b085b98 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:28.758439 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2051dc-b0df-4849-84b7-e7de6990e9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:28.759344 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2051dc-b0df-4849-84b7-e7de6990e9e6 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:28.759730 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2051dc-b0df-4849-84b7-e7de6990e9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:28.760667 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2051dc-b0df-4849-84b7-e7de6990e9e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:28.771765 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2051dc-b0df-4849-84b7-e7de6990e9e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:28.773356 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2051dc-b0df-4849-84b7-e7de6990e9e6 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:28.789027 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-6e2051dc-b0df-4849-84b7-e7de6990e9e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:28.789027 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:28.789027 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:28.789027 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:28.789027 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:28.789027 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:28.789027 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:28.789027 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:28.789027 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:28.789027 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:28.789027 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:28.789027 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:28.789027 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:28.789027 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:28.789027 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:28.789027 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:28.790178 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:28.790178 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:28.790178 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:28.790178 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:28.790178 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:28.790178 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:28.790178 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:28.790178 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:28.790178 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:28.790178 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:28.790178 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:28.790178 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:28.790178 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:28.790178 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:28.790178 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:28.790178 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:28.790178 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:28.791331 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:28.791331 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:28.791331 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:28.794328 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 389/775] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:28.796648 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f3ee0e-ce6b-4b3a-9da0-da14ea785ec1 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:28.800494 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2051dc-b0df-4849-84b7-e7de6990e9e6 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:28.832608 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dc90a1-90c4-4e98-a1a1-f15d0d904ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:28.833550 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc90a1-90c4-4e98-a1a1-f15d0d904ad0 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:28.834013 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc90a1-90c4-4e98-a1a1-f15d0d904ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:28.834464 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc90a1-90c4-4e98-a1a1-f15d0d904ad0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:28.846315 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dc90a1-90c4-4e98-a1a1-f15d0d904ad0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:28.848018 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dc90a1-90c4-4e98-a1a1-f15d0d904ad0 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:28.863180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-24dc90a1-90c4-4e98-a1a1-f15d0d904ad0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:28.863180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:28.863180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:28.863180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:28.863180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:28.863180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:28.863180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:28.863180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:28.863180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:28.863180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:28.863180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:28.863180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:28.863180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:28.863180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:28.863180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:28.863180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:28.864180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:28.864180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:28.864180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:28.864180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:28.864180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:28.864180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:28.864180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:28.864180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:28.864180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:28.864180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:28.864180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:28.864180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:28.864180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:28.864180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:28.864180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:28.864180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:28.864180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:28.865052 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:28.865052 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:28.865052 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:28.867062 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 387/776] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:28.882532 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dc90a1-90c4-4e98-a1a1-f15d0d904ad0 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:28.919999 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2051dc-b0df-4849-84b7-e7de6990e9e6 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:28.925095 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:28.950721 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7750f60-c4b8-430e-814a-22bac212d04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:28.951409 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7750f60-c4b8-430e-814a-22bac212d04d None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:28.951684 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7750f60-c4b8-430e-814a-22bac212d04d None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:28.951979 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7750f60-c4b8-430e-814a-22bac212d04d None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:28.952797 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7750f60-c4b8-430e-814a-22bac212d04d None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:28.953107 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7750f60-c4b8-430e-814a-22bac212d04d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:28.954479 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7750f60-c4b8-430e-814a-22bac212d04d None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:28.971992 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 390/777] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:28.971992 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c68de7-786a-433a-bf2f-7a685632cb4d None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:28.971992 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c68de7-786a-433a-bf2f-7a685632cb4d None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:28.971992 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c68de7-786a-433a-bf2f-7a685632cb4d None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:28.971992 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 391/778] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:28.971992 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c68de7-786a-433a-bf2f-7a685632cb4d None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:29.005049 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:29.031915 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dc90a1-90c4-4e98-a1a1-f15d0d904ad0 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:29.069342 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9f96b-40c9-4bf1-af1b-8c6d6cb15067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:29.070063 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9f96b-40c9-4bf1-af1b-8c6d6cb15067 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:29.070414 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9f96b-40c9-4bf1-af1b-8c6d6cb15067 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:29.070757 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9f96b-40c9-4bf1-af1b-8c6d6cb15067 None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:29.071479 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a9f96b-40c9-4bf1-af1b-8c6d6cb15067 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:29.071851 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a9f96b-40c9-4bf1-af1b-8c6d6cb15067 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:29.073850 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a9f96b-40c9-4bf1-af1b-8c6d6cb15067 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:29.085685 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 388/779] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:29.086113 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c68de7-786a-433a-bf2f-7a685632cb4d None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:29.086851 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26432607-cbf2-44a3-ac6d-42a392249f49 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:29.087138 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26432607-cbf2-44a3-ac6d-42a392249f49 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:29.087411 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26432607-cbf2-44a3-ac6d-42a392249f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:29.105882 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-26432607-cbf2-44a3-ac6d-42a392249f49 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:29.106994 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-26432607-cbf2-44a3-ac6d-42a392249f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:29.126812 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b7aec-2e0a-4763-8835-36a96428be8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:29.127618 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b7aec-2e0a-4763-8835-36a96428be8b None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:29.128109 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b7aec-2e0a-4763-8835-36a96428be8b None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:29.128399 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b7aec-2e0a-4763-8835-36a96428be8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:29.135723 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0b7aec-2e0a-4763-8835-36a96428be8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:29.137176 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0b7aec-2e0a-4763-8835-36a96428be8b None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:29.148490 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-0e0b7aec-2e0a-4763-8835-36a96428be8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:29.148490 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:29.148490 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:29.148490 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:29.148490 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:29.148490 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:29.148490 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:29.148490 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:29.148490 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:29.148490 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:29.148490 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:29.148490 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:29.148490 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 23:31:29.148490 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 23:31:29.148490 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 23:31:29.149457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 23:31:29.149457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:29.149457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:29.149457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:29.149457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:29.149457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:29.149457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:29.149457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:29.149457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:29.149457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:29.149457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:29.149457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:29.149457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:29.149457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:29.149457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:29.149457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:29.149457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:29.150333 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 23:31:29.150333 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 23:31:29.150333 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 23:31:29.150333 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 23:31:29.150333 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 23:31:29.150333 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 23:31:29.150333 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:29.150333 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:29.150333 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 392/780] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:31:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:29.150333 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b7aec-2e0a-4763-8835-36a96428be8b None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:29.181359 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:29.183058 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 389/781] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:31:29.196557 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26432607-cbf2-44a3-ac6d-42a392249f49 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:29.257223 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:29.275004 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b7aec-2e0a-4763-8835-36a96428be8b None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:29.313496 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26432607-cbf2-44a3-ac6d-42a392249f49 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:29.318712 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdda5fd-9a39-4f19-9184-62a4a66d17bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:29.319403 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdda5fd-9a39-4f19-9184-62a4a66d17bf None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:29.319686 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdda5fd-9a39-4f19-9184-62a4a66d17bf None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:29.319980 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdda5fd-9a39-4f19-9184-62a4a66d17bf None admin] PATH_INFO: `/v3/services/network` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:29.320563 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdda5fd-9a39-4f19-9184-62a4a66d17bf None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:29.321930 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdda5fd-9a39-4f19-9184-62a4a66d17bf None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:29.330939 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-4bdda5fd-9a39-4f19-9184-62a4a66d17bf None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 17 23:31:29.330939 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:29.330939 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 23:31:29.330939 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 23:31:29.330939 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 23:31:29.330939 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 23:31:29.330939 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 23:31:29.330939 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:29.330939 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 17 23:31:29.330939 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:29.330939 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 23:31:29.330939 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:29.330939 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:29.330939 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:29.330939 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:29.330939 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:29.330939 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:29.332180 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:29.332180 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:29.332180 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:29.332180 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:29.332180 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:29.332180 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:29.332180 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 23:31:29.332180 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 23:31:29.332180 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 23:31:29.332180 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 23:31:29.332180 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:29.332180 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:29.332180 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:29.332180 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:29.332180 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:29.332180 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:29.333127 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:29.333127 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:29.333127 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:29.333127 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:29.333127 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:29.333127 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:29.333127 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:29.333127 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:29.333127 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:29.333127 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:29.333127 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 23:31:29.333127 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:29.333127 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 17 23:31:29.333127 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:29.336926 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 393/782] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:31:29 2022] GET /identity/v3/services/network => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:29.337735 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdda5fd-9a39-4f19-9184-62a4a66d17bf None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:29.361179 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da20b930-58aa-43fb-b25c-fb7cdc4e5e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:29.362528 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20b930-58aa-43fb-b25c-fb7cdc4e5e85 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:29.362972 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20b930-58aa-43fb-b25c-fb7cdc4e5e85 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:29.363666 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20b930-58aa-43fb-b25c-fb7cdc4e5e85 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:29.374015 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da20b930-58aa-43fb-b25c-fb7cdc4e5e85 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:29.376726 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da20b930-58aa-43fb-b25c-fb7cdc4e5e85 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:29.393415 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-da20b930-58aa-43fb-b25c-fb7cdc4e5e85 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 17 23:31:29.393415 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:29.393415 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:29.393415 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:29.393415 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:29.393415 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:29.393415 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:29.393415 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:29.393415 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:29.393415 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:29.393415 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:29.393415 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:29.393415 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 23:31:29.393415 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 23:31:29.393415 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 23:31:29.393415 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 23:31:29.394585 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:29.394585 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:29.394585 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 23:31:29.394585 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:29.394585 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 23:31:29.394585 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:29.394585 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:29.394585 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:29.394585 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:29.394585 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:29.394585 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:29.394585 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:29.394585 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:29.394585 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:29.394585 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:29.394585 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:29.394585 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:29.394585 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:29.395759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:29.395759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:29.395759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 23:31:29.395759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 23:31:29.395759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 23:31:29.395759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 23:31:29.395759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 23:31:29.395759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 23:31:29.395759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 17 23:31:29.395759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:29.398689 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 390/783] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Aug 17 23:31:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:29.399878 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da20b930-58aa-43fb-b25c-fb7cdc4e5e85 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:29.446520 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:29.450681 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdda5fd-9a39-4f19-9184-62a4a66d17bf None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:29.486368 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d4364f-6c0c-4b48-aa58-b259dda56dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:29.486973 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4364f-6c0c-4b48-aa58-b259dda56dd5 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:29.487251 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4364f-6c0c-4b48-aa58-b259dda56dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:29.487526 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4364f-6c0c-4b48-aa58-b259dda56dd5 None admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:29.488263 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d4364f-6c0c-4b48-aa58-b259dda56dd5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:29.488569 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d4364f-6c0c-4b48-aa58-b259dda56dd5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:29.490028 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d4364f-6c0c-4b48-aa58-b259dda56dd5 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:29.503039 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 394/784] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 17 23:31:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:29.503992 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d4364f-6c0c-4b48-aa58-b259dda56dd5 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:29.519220 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da20b930-58aa-43fb-b25c-fb7cdc4e5e85 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:29.544622 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:29.557081 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf9c111-4352-492e-93ea-f26aa65ab449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:29.557693 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9c111-4352-492e-93ea-f26aa65ab449 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:29.557971 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9c111-4352-492e-93ea-f26aa65ab449 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:29.558245 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9c111-4352-492e-93ea-f26aa65ab449 None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:29.558877 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf9c111-4352-492e-93ea-f26aa65ab449 None admin] RBAC: Adding query filter params (name=network) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:29.559181 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf9c111-4352-492e-93ea-f26aa65ab449 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:29.560688 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf9c111-4352-492e-93ea-f26aa65ab449 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:29.569646 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 391/785] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Aug 17 23:31:29 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:29.569646 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf9c111-4352-492e-93ea-f26aa65ab449 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:29.620035 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d4364f-6c0c-4b48-aa58-b259dda56dd5 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:29.655599 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8effb1ec-0cc1-48fb-b828-9ff7ac1711e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:29.656236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8effb1ec-0cc1-48fb-b828-9ff7ac1711e1 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:29.656517 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8effb1ec-0cc1-48fb-b828-9ff7ac1711e1 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:29.656785 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8effb1ec-0cc1-48fb-b828-9ff7ac1711e1 None admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:29.657537 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8effb1ec-0cc1-48fb-b828-9ff7ac1711e1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:29.657847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8effb1ec-0cc1-48fb-b828-9ff7ac1711e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:29.659267 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8effb1ec-0cc1-48fb-b828-9ff7ac1711e1 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:29.670926 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf9c111-4352-492e-93ea-f26aa65ab449 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:29.678497 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 395/786] 10.222.0.26 () {56 vars in 1149 bytes} [Wed Aug 17 23:31:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:29.679124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8effb1ec-0cc1-48fb-b828-9ff7ac1711e1 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:29.693645 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:29.712923 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ff4f4-2156-4c2b-b06d-421f1d0c4de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:29.713584 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ff4f4-2156-4c2b-b06d-421f1d0c4de7 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:29.713862 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ff4f4-2156-4c2b-b06d-421f1d0c4de7 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:29.714136 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ff4f4-2156-4c2b-b06d-421f1d0c4de7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:29.728454 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591ff4f4-2156-4c2b-b06d-421f1d0c4de7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:31:29.734978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-591ff4f4-2156-4c2b-b06d-421f1d0c4de7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:31:29.734978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:29.734978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:29.734978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:29.734978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:29.734978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:29.734978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:29.734978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:29.734978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:29.734978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:29.734978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:29.734978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:29.734978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:31:29.734978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:31:29.734978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:31:29.734978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:31:29.736157 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:31:29.736157 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:31:29.736157 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:31:29.736157 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:31:29.736157 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:29.736157 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:29.736157 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:29.736157 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:29.736157 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:29.736157 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:29.736157 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:29.736157 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:29.736157 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:29.736157 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:29.736157 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:29.736157 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:29.737011 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:29.737011 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:29.737011 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:29.737011 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:29.737011 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:31:29.737011 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:31:29.737011 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:31:29.737011 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:31:29.737011 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:31:29.737011 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:31:29.737011 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:31:29.737011 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:29.737011 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 392/787] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 17 23:31:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:29.737711 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ff4f4-2156-4c2b-b06d-421f1d0c4de7 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:29.789279 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8effb1ec-0cc1-48fb-b828-9ff7ac1711e1 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:29.792831 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:29.828674 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989af7b2-f820-43db-93ef-fd8375405c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:29.828674 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989af7b2-f820-43db-93ef-fd8375405c91 None admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:29.828674 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989af7b2-f820-43db-93ef-fd8375405c91 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:29.828674 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989af7b2-f820-43db-93ef-fd8375405c91 None admin] PATH_INFO: `/v3/projects/4de86915ef4340248ba8a8f51d56594e/users/ec1dcbe43b0749c3bcc4af9100d771d1/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:29.829757 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989af7b2-f820-43db-93ef-fd8375405c91 None admin] RBAC: Authorizing `identity:create_grant(project_id=4de86915ef4340248ba8a8f51d56594e, user_id=ec1dcbe43b0749c3bcc4af9100d771d1, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:29.831173 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989af7b2-f820-43db-93ef-fd8375405c91 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:29.843753 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 396/788] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 17 23:31:29 2022] PUT /identity/v3/projects/4de86915ef4340248ba8a8f51d56594e/users/ec1dcbe43b0749c3bcc4af9100d771d1/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:31:29.844649 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989af7b2-f820-43db-93ef-fd8375405c91 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:29.871400 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ff4f4-2156-4c2b-b06d-421f1d0c4de7 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:29.921633 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb81a9-369b-445b-9f2d-ac40d4b1cbda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:29.922924 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb81a9-369b-445b-9f2d-ac40d4b1cbda None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:29.923438 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb81a9-369b-445b-9f2d-ac40d4b1cbda None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:29.923943 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb81a9-369b-445b-9f2d-ac40d4b1cbda None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:29.924988 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb81a9-369b-445b-9f2d-ac40d4b1cbda None admin] RBAC: Adding query filter params (type=network) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:29.926393 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb81a9-369b-445b-9f2d-ac40d4b1cbda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:29.928069 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb81a9-369b-445b-9f2d-ac40d4b1cbda None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:29.935825 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 393/789] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Aug 17 23:31:29 2022] GET /identity/v3/services?type=network => generated 350 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:29.937203 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb81a9-369b-445b-9f2d-ac40d4b1cbda None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:29.942481 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:29.965555 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989af7b2-f820-43db-93ef-fd8375405c91 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:30.011237 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f498862a-eeb8-49c7-b7a4-9120ef51ec31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:30.011878 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498862a-eeb8-49c7-b7a4-9120ef51ec31 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:30.012235 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498862a-eeb8-49c7-b7a4-9120ef51ec31 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:30.012508 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498862a-eeb8-49c7-b7a4-9120ef51ec31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:30.024664 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f498862a-eeb8-49c7-b7a4-9120ef51ec31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:30.026755 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f498862a-eeb8-49c7-b7a4-9120ef51ec31 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:30.046124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-f498862a-eeb8-49c7-b7a4-9120ef51ec31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:30.046124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:30.046124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:30.046124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:30.046124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:30.046124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:30.046124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:30.046124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:30.046124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:30.046124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:30.046124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:30.046124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:30.046124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:30.046124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:30.046124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:30.046124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:30.047134 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:30.047134 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:30.047134 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:30.047134 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:30.047134 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:30.047134 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:30.047134 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:30.047134 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:30.047134 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:30.047134 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:30.047134 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:30.047134 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:30.047134 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:30.047134 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:30.047134 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:30.047134 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:30.047134 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:30.048056 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:30.048056 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:30.048056 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:30.048056 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 397/790] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:30.048786 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f498862a-eeb8-49c7-b7a4-9120ef51ec31 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:30.082490 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb81a9-369b-445b-9f2d-ac40d4b1cbda None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:30.096698 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:30.130537 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da76e8-b8e3-48ae-b7e7-c2b09575f2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:30.131131 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da76e8-b8e3-48ae-b7e7-c2b09575f2e6 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:30.131378 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da76e8-b8e3-48ae-b7e7-c2b09575f2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:30.131584 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da76e8-b8e3-48ae-b7e7-c2b09575f2e6 None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:30.132284 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da76e8-b8e3-48ae-b7e7-c2b09575f2e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:30.132585 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da76e8-b8e3-48ae-b7e7-c2b09575f2e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:30.134401 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da76e8-b8e3-48ae-b7e7-c2b09575f2e6 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:30.144728 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 394/791] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:31:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:30.145551 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da76e8-b8e3-48ae-b7e7-c2b09575f2e6 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:30.164056 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f498862a-eeb8-49c7-b7a4-9120ef51ec31 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:30.187340 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:30.199904 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6061dd31-77ce-489c-860d-897ce6e807f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:30.200639 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061dd31-77ce-489c-860d-897ce6e807f9 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:30.200819 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061dd31-77ce-489c-860d-897ce6e807f9 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:30.201091 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061dd31-77ce-489c-860d-897ce6e807f9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:30.201797 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6061dd31-77ce-489c-860d-897ce6e807f9 None admin] RBAC: Adding query filter params (interface=public, service_id=465ac52d2426484088c1130f89643e94, region_id=RegionOne) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:30.202108 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6061dd31-77ce-489c-860d-897ce6e807f9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:30.203467 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6061dd31-77ce-489c-860d-897ce6e807f9 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:30.209239 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 398/792] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Aug 17 23:31:30 2022] GET /identity/v3/endpoints?service_id=465ac52d2426484088c1130f89643e94&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:30.210693 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28229e5c-b3a2-4d69-b054-a86c192e7e71 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:30.211164 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28229e5c-b3a2-4d69-b054-a86c192e7e71 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:30.211633 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28229e5c-b3a2-4d69-b054-a86c192e7e71 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:30.213064 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 399/793] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:30.214009 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28229e5c-b3a2-4d69-b054-a86c192e7e71 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:30.268420 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da76e8-b8e3-48ae-b7e7-c2b09575f2e6 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:30.303575 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81def6-a763-4c9f-b686-9ac4a0062ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:30.304217 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81def6-a763-4c9f-b686-9ac4a0062ab6 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:30.304631 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81def6-a763-4c9f-b686-9ac4a0062ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:30.304801 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81def6-a763-4c9f-b686-9ac4a0062ab6 None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:30.305474 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a81def6-a763-4c9f-b686-9ac4a0062ab6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:30.305940 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a81def6-a763-4c9f-b686-9ac4a0062ab6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:30.307733 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a81def6-a763-4c9f-b686-9ac4a0062ab6 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:30.324775 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 395/794] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:30.324775 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81def6-a763-4c9f-b686-9ac4a0062ab6 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:30.326648 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28229e5c-b3a2-4d69-b054-a86c192e7e71 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:30.357233 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:30.362320 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99d224-c195-4485-a370-bd34f0e47ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:30.362951 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99d224-c195-4485-a370-bd34f0e47ccc None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:30.363228 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99d224-c195-4485-a370-bd34f0e47ccc None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:30.363501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99d224-c195-4485-a370-bd34f0e47ccc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:30.374658 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99d224-c195-4485-a370-bd34f0e47ccc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:30.376263 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99d224-c195-4485-a370-bd34f0e47ccc None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:30.391072 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-8c99d224-c195-4485-a370-bd34f0e47ccc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:30.391072 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:30.391072 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:30.391072 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:30.391072 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:30.391072 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:30.391072 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:30.391072 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:30.391072 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:30.391072 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:30.391072 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:30.391072 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:30.391072 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:30.391072 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:30.391072 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:30.391072 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:30.392037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:30.392037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:30.392037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:30.392037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:30.392037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:30.392037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:30.392037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:30.392037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:30.392037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:30.392037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:30.392037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:30.392037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:30.392037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:30.392037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:30.392037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:30.392037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:30.392037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:30.392982 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:30.392982 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:30.392982 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:30.392982 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 400/795] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:30.393759 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c4c8b-7195-490a-970e-60418feff8e0 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:30.393759 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c4c8b-7195-490a-970e-60418feff8e0 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:30.393920 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c4c8b-7195-490a-970e-60418feff8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:30.412630 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-677c4c8b-7195-490a-970e-60418feff8e0 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:30.413832 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-677c4c8b-7195-490a-970e-60418feff8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:30.437173 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81def6-a763-4c9f-b686-9ac4a0062ab6 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:30.453649 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:30.479939 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38517a1a-a479-4cf2-bf3e-eb803a05121b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:30.480397 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38517a1a-a479-4cf2-bf3e-eb803a05121b None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:30.480668 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38517a1a-a479-4cf2-bf3e-eb803a05121b None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:30.480966 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38517a1a-a479-4cf2-bf3e-eb803a05121b None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:30.481542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38517a1a-a479-4cf2-bf3e-eb803a05121b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:30.482904 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38517a1a-a479-4cf2-bf3e-eb803a05121b None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:30.487893 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 401/796] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:31:30.488769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c4c8b-7195-490a-970e-60418feff8e0 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:30.504024 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 396/797] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Aug 17 23:31:30 2022] GET /identity/v3/services => generated 772 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:30.504024 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38517a1a-a479-4cf2-bf3e-eb803a05121b None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:30.687827 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:30.690447 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c4c8b-7195-490a-970e-60418feff8e0 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:30.700891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38517a1a-a479-4cf2-bf3e-eb803a05121b None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:30.734447 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f64509-5bfb-4442-b11b-22b9eef913b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:30.735513 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f64509-5bfb-4442-b11b-22b9eef913b1 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:30.736291 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f64509-5bfb-4442-b11b-22b9eef913b1 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:30.736864 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f64509-5bfb-4442-b11b-22b9eef913b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:30.741093 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59cd65-426b-48db-bbe3-6812805a404b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:30.741806 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59cd65-426b-48db-bbe3-6812805a404b None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:30.742156 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59cd65-426b-48db-bbe3-6812805a404b None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:30.742499 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59cd65-426b-48db-bbe3-6812805a404b None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:30.743218 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59cd65-426b-48db-bbe3-6812805a404b None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:30.743590 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59cd65-426b-48db-bbe3-6812805a404b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:30.745130 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59cd65-426b-48db-bbe3-6812805a404b None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:30.746076 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f64509-5bfb-4442-b11b-22b9eef913b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:30.748369 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f64509-5bfb-4442-b11b-22b9eef913b1 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:30.758215 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 397/798] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:30.759413 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59cd65-426b-48db-bbe3-6812805a404b None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:30.767685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-d7f64509-5bfb-4442-b11b-22b9eef913b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:30.767685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:30.767685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:30.767685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:30.767685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:30.767685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:30.767685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:30.767685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:30.767685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:30.767685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:30.767685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:30.767685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:30.767685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 23:31:30.767685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 23:31:30.767685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 23:31:30.768678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 23:31:30.768678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:30.768678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:30.768678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:30.768678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:30.768678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:30.768678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:30.768678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:30.768678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:30.768678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:30.768678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:30.768678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:30.768678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:30.768678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:30.768678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:30.768678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:30.768678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:30.769624 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 23:31:30.769624 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 23:31:30.769624 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 23:31:30.769624 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 23:31:30.769624 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 23:31:30.769624 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 23:31:30.769624 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:30.769624 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:30.769624 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 402/799] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:31:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:30.772338 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f64509-5bfb-4442-b11b-22b9eef913b1 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:30.807748 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:30.875872 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59cd65-426b-48db-bbe3-6812805a404b None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:30.893005 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f64509-5bfb-4442-b11b-22b9eef913b1 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:30.919455 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191006a0-8bb9-487e-b320-5fff485ba59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:30.920529 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191006a0-8bb9-487e-b320-5fff485ba59f None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:30.920836 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191006a0-8bb9-487e-b320-5fff485ba59f None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:30.921112 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191006a0-8bb9-487e-b320-5fff485ba59f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:30.921974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191006a0-8bb9-487e-b320-5fff485ba59f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:30.924036 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191006a0-8bb9-487e-b320-5fff485ba59f None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:30.931921 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb88583-1588-4cb4-88c7-0eb366481acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:30.932685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb88583-1588-4cb4-88c7-0eb366481acd None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:30.933070 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb88583-1588-4cb4-88c7-0eb366481acd None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:30.933443 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb88583-1588-4cb4-88c7-0eb366481acd None admin] PATH_INFO: `/v3/users/swift` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:30.934321 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-191006a0-8bb9-487e-b320-5fff485ba59f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 17 23:31:30.934321 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:30.934321 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 23:31:30.934321 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 23:31:30.934321 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 23:31:30.934321 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 23:31:30.934321 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 23:31:30.934321 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:30.934321 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 17 23:31:30.934321 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:30.934321 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 23:31:30.934321 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:30.934321 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:30.934321 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:30.934321 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:30.934321 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:30.934321 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:30.935375 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:30.935375 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:30.935375 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:30.935375 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:30.935375 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:30.935375 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:30.935375 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 23:31:30.935375 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 23:31:30.935375 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 23:31:30.935375 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 23:31:30.935375 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:30.935375 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:30.935375 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:30.935375 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:30.935375 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:30.935375 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:30.936260 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:30.936260 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:30.936260 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:30.936260 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:30.936260 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:30.936260 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:30.936260 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:30.936260 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:30.936260 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:30.936260 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:30.936260 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 23:31:30.936260 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:30.936260 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 17 23:31:30.936260 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:30.940179 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 398/800] 10.222.0.26 () {56 vars in 1129 bytes} [Wed Aug 17 23:31:30 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:30.941029 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191006a0-8bb9-487e-b320-5fff485ba59f None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:30.942345 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb88583-1588-4cb4-88c7-0eb366481acd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:30.943810 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb88583-1588-4cb4-88c7-0eb366481acd None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:30.955909 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-5cb88583-1588-4cb4-88c7-0eb366481acd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 17 23:31:30.955909 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:30.955909 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:30.955909 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:30.955909 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:30.955909 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:30.955909 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:30.955909 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:30.955909 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:30.955909 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:30.955909 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:30.955909 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:30.955909 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 23:31:30.955909 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 23:31:30.955909 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 23:31:30.955909 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 23:31:30.958053 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:30.958053 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:30.958053 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 23:31:30.958053 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:30.958053 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 23:31:30.958053 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:30.958053 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:30.958053 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:30.958053 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:30.958053 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:30.958053 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:30.958053 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:30.958053 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:30.958053 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:30.958053 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:30.958053 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:30.958053 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:30.958053 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:30.960430 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:30.960430 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:30.960430 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 23:31:30.960430 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 23:31:30.960430 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 23:31:30.960430 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 23:31:30.960430 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 23:31:30.960430 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 23:31:30.960430 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 17 23:31:30.960430 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:30.960430 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 403/801] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 17 23:31:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:30.960430 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb88583-1588-4cb4-88c7-0eb366481acd None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:30.968399 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:31.048467 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191006a0-8bb9-487e-b320-5fff485ba59f None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:31.056348 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:31.067936 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb88583-1588-4cb4-88c7-0eb366481acd None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:31.084509 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d361ec9-23f2-42f6-9bfd-b6c6a2d25cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:31.085246 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d361ec9-23f2-42f6-9bfd-b6c6a2d25cba None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:31.085657 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d361ec9-23f2-42f6-9bfd-b6c6a2d25cba None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:31.086037 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d361ec9-23f2-42f6-9bfd-b6c6a2d25cba None admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:31.086825 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d361ec9-23f2-42f6-9bfd-b6c6a2d25cba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:31.087238 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d361ec9-23f2-42f6-9bfd-b6c6a2d25cba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:31.089341 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d361ec9-23f2-42f6-9bfd-b6c6a2d25cba None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:31.106274 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 399/802] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 17 23:31:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:31.106274 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d361ec9-23f2-42f6-9bfd-b6c6a2d25cba None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:31.110804 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78469f99-25e3-4343-affb-d1124767528a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:31.111778 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78469f99-25e3-4343-affb-d1124767528a None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:31.111778 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78469f99-25e3-4343-affb-d1124767528a None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:31.112148 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78469f99-25e3-4343-affb-d1124767528a None admin] PATH_INFO: `/v3/services` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:31.112610 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78469f99-25e3-4343-affb-d1124767528a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:31.112916 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78469f99-25e3-4343-affb-d1124767528a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:31.114294 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78469f99-25e3-4343-affb-d1124767528a None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:31.126196 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 404/803] 10.222.0.26 () {56 vars in 1124 bytes} [Wed Aug 17 23:31:30 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:31.126985 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78469f99-25e3-4343-affb-d1124767528a None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:31.200286 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:31.222917 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d361ec9-23f2-42f6-9bfd-b6c6a2d25cba None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:31.234987 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78469f99-25e3-4343-affb-d1124767528a None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:31.261892 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a141df-1c18-45ef-96ab-0964422a2261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:31.262738 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a141df-1c18-45ef-96ab-0964422a2261 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:31.263220 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a141df-1c18-45ef-96ab-0964422a2261 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:31.263638 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a141df-1c18-45ef-96ab-0964422a2261 None admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:31.264727 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a141df-1c18-45ef-96ab-0964422a2261 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:31.265206 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a141df-1c18-45ef-96ab-0964422a2261 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:31.266800 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a141df-1c18-45ef-96ab-0964422a2261 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:31.270170 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb32f0-9d01-488e-bffe-c9c09ae61aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:31.270537 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb32f0-9d01-488e-bffe-c9c09ae61aae None admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:31.270825 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb32f0-9d01-488e-bffe-c9c09ae61aae None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:31.271103 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb32f0-9d01-488e-bffe-c9c09ae61aae None admin] PATH_INFO: `/v3/projects/4de86915ef4340248ba8a8f51d56594e/users/d0a13dbffb4a49508b7fb661cf84e803/roles/b64f80e6f3ea4ad49b94acdde42ccd87` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:31.286831 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 400/804] 10.222.0.26 () {56 vars in 1135 bytes} [Wed Aug 17 23:31:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:31.287029 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb32f0-9d01-488e-bffe-c9c09ae61aae None admin] RBAC: Authorizing `identity:create_grant(project_id=4de86915ef4340248ba8a8f51d56594e, user_id=d0a13dbffb4a49508b7fb661cf84e803, role_id=b64f80e6f3ea4ad49b94acdde42ccd87)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:31.288199 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a141df-1c18-45ef-96ab-0964422a2261 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:31.288648 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb32f0-9d01-488e-bffe-c9c09ae61aae None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:31.312373 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 405/805] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 17 23:31:31 2022] PUT /identity/v3/projects/4de86915ef4340248ba8a8f51d56594e/users/d0a13dbffb4a49508b7fb661cf84e803/roles/b64f80e6f3ea4ad49b94acdde42ccd87 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:31:31.314036 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb32f0-9d01-488e-bffe-c9c09ae61aae None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:31.318308 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:31.411954 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a141df-1c18-45ef-96ab-0964422a2261 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:31.502219 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c079a3-bf3d-45b6-9e3f-23eaf548e135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:31.503411 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c079a3-bf3d-45b6-9e3f-23eaf548e135 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:31.504585 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c079a3-bf3d-45b6-9e3f-23eaf548e135 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:31.505241 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c079a3-bf3d-45b6-9e3f-23eaf548e135 None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:31.506896 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c079a3-bf3d-45b6-9e3f-23eaf548e135 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:31.507555 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c079a3-bf3d-45b6-9e3f-23eaf548e135 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:31.511085 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c079a3-bf3d-45b6-9e3f-23eaf548e135 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:31.524004 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 401/806] 10.222.0.26 () {56 vars in 1124 bytes} [Wed Aug 17 23:31:31 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:31.526265 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c079a3-bf3d-45b6-9e3f-23eaf548e135 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:31.530167 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:31.547456 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb32f0-9d01-488e-bffe-c9c09ae61aae None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:31.607346 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e958f7-59ca-4170-a8c0-aad49b2f1adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:31.608006 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e958f7-59ca-4170-a8c0-aad49b2f1adf None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:31.608326 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e958f7-59ca-4170-a8c0-aad49b2f1adf None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:31.608601 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e958f7-59ca-4170-a8c0-aad49b2f1adf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:31.626510 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e958f7-59ca-4170-a8c0-aad49b2f1adf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:31.627990 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e958f7-59ca-4170-a8c0-aad49b2f1adf None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:31.659228 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c079a3-bf3d-45b6-9e3f-23eaf548e135 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:31.669002 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-52e958f7-59ca-4170-a8c0-aad49b2f1adf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:31.669002 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:31.669002 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:31.669002 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:31.669002 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:31.669002 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:31.669002 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:31.669002 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:31.669002 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:31.669002 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:31.669002 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:31.669002 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:31.669002 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:31.669002 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:31.669002 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:31.669002 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:31.670001 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:31.670001 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:31.670001 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:31.670001 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:31.670001 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:31.670001 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:31.670001 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:31.670001 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:31.670001 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:31.670001 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:31.670001 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:31.670001 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:31.670001 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:31.670001 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:31.670001 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:31.670001 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:31.670001 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:31.671468 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:31.671468 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:31.671468 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:31.671468 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 406/807] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:31.676854 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e958f7-59ca-4170-a8c0-aad49b2f1adf None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:31.711250 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910a672-9f4f-4d61-a6ed-8378c0d9d63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:31.711791 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910a672-9f4f-4d61-a6ed-8378c0d9d63d None admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:31.712092 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910a672-9f4f-4d61-a6ed-8378c0d9d63d None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:31.712367 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910a672-9f4f-4d61-a6ed-8378c0d9d63d None admin] PATH_INFO: `/v3/endpoints` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:31.713718 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a910a672-9f4f-4d61-a6ed-8378c0d9d63d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:31.719993 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a910a672-9f4f-4d61-a6ed-8378c0d9d63d None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:31.742200 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 402/808] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:31:31 2022] POST /identity/v3/endpoints => generated 346 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:31:31.814610 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:31.860223 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e0716-4d23-4f8b-937b-3fdc3bdcd70d None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:31.860538 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e0716-4d23-4f8b-937b-3fdc3bdcd70d None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:31.860780 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e0716-4d23-4f8b-937b-3fdc3bdcd70d None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:31.862360 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 403/809] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 23:31:31.867556 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1aa44-ee3b-4489-a949-09640f3f76a0 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:31.867843 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1aa44-ee3b-4489-a949-09640f3f76a0 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:31.868173 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1aa44-ee3b-4489-a949-09640f3f76a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:31.880912 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e958f7-59ca-4170-a8c0-aad49b2f1adf None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:31.913188 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-49c1aa44-ee3b-4489-a949-09640f3f76a0 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:31.914426 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-49c1aa44-ee3b-4489-a949-09640f3f76a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:31.932786 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c66f93-2759-42ef-a37f-318325c4a9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:31.933821 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c66f93-2759-42ef-a37f-318325c4a9b8 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:31.934346 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c66f93-2759-42ef-a37f-318325c4a9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:31.934817 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c66f93-2759-42ef-a37f-318325c4a9b8 None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:31.936322 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c66f93-2759-42ef-a37f-318325c4a9b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:31.936900 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c66f93-2759-42ef-a37f-318325c4a9b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:31.938706 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c66f93-2759-42ef-a37f-318325c4a9b8 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:31.955560 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 407/810] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:31.961432 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c66f93-2759-42ef-a37f-318325c4a9b8 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:32.000397 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:32.007761 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 404/811] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:31:32.074485 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c66f93-2759-42ef-a37f-318325c4a9b8 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:32.088623 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:32.109788 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d60472-9c2d-4cda-9903-7d32f2bb644a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:32.110627 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d60472-9c2d-4cda-9903-7d32f2bb644a None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:32.111044 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d60472-9c2d-4cda-9903-7d32f2bb644a None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:32.111477 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d60472-9c2d-4cda-9903-7d32f2bb644a None admin] PATH_INFO: `/v3/projects/service` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:32.118518 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d60472-9c2d-4cda-9903-7d32f2bb644a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:32.120706 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d60472-9c2d-4cda-9903-7d32f2bb644a None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:32.132117 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-b3d60472-9c2d-4cda-9903-7d32f2bb644a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:32.132117 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:32.132117 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:32.132117 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:32.132117 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:32.132117 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:32.132117 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:32.132117 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:32.132117 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:32.132117 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:32.132117 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:32.132117 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:32.132117 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 23:31:32.132117 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 23:31:32.132117 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 23:31:32.133189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 23:31:32.133189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:32.133189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:32.133189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:32.133189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:32.133189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:32.133189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:32.133189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:32.133189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:32.133189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:32.133189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:32.133189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:32.133189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:32.133189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:32.133189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:32.133189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:32.133189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:32.134258 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 23:31:32.134258 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 23:31:32.134258 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 23:31:32.134258 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 23:31:32.134258 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 23:31:32.134258 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 23:31:32.134258 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:32.134258 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:32.136819 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 408/812] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:31:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:31:32.142686 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c1aa44-ee3b-4489-a949-09640f3f76a0 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:32.248982 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c1aa44-ee3b-4489-a949-09640f3f76a0 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:32.293568 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e47bc-876d-48c0-96ae-eb7c863d0b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:32.294476 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e47bc-876d-48c0-96ae-eb7c863d0b71 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:32.294924 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e47bc-876d-48c0-96ae-eb7c863d0b71 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:32.295365 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e47bc-876d-48c0-96ae-eb7c863d0b71 None admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:32.296289 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7e47bc-876d-48c0-96ae-eb7c863d0b71 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:32.296764 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7e47bc-876d-48c0-96ae-eb7c863d0b71 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:32.298471 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7e47bc-876d-48c0-96ae-eb7c863d0b71 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:32.298591 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d60472-9c2d-4cda-9903-7d32f2bb644a None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:32.313158 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 405/813] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 17 23:31:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:32.319501 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e47bc-876d-48c0-96ae-eb7c863d0b71 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:32.407719 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:32.433053 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d60472-9c2d-4cda-9903-7d32f2bb644a None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:32.482097 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f6498-7133-4125-b855-9d199d287599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:32.482980 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f6498-7133-4125-b855-9d199d287599 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:32.483425 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f6498-7133-4125-b855-9d199d287599 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:32.483840 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f6498-7133-4125-b855-9d199d287599 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:32.503875 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409f6498-7133-4125-b855-9d199d287599 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:31:32.510851 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-409f6498-7133-4125-b855-9d199d287599 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:31:32.510851 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:32.510851 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:32.510851 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:32.510851 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:32.510851 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:32.510851 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:32.510851 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:32.510851 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:32.510851 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:32.510851 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:32.510851 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:32.510851 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:31:32.510851 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:31:32.510851 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:31:32.510851 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:31:32.511992 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:31:32.511992 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:31:32.511992 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:31:32.511992 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:31:32.511992 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:32.511992 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:32.511992 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:32.511992 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:32.511992 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:32.511992 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:32.511992 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:32.511992 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:32.511992 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:32.511992 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:32.511992 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:32.511992 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:32.513013 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:32.513013 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:32.513013 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:32.513013 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:32.513013 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:31:32.513013 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:31:32.513013 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:31:32.513013 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:31:32.513013 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:31:32.513013 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:31:32.513013 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:31:32.513013 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:32.521599 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 409/814] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 17 23:31:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:31:32.522854 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e47bc-876d-48c0-96ae-eb7c863d0b71 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:32.528159 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f6498-7133-4125-b855-9d199d287599 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:32.579587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282526ae-2d97-4773-90da-f2bc3bee2f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:32.580524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282526ae-2d97-4773-90da-f2bc3bee2f30 None admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:32.580969 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282526ae-2d97-4773-90da-f2bc3bee2f30 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:32.581380 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282526ae-2d97-4773-90da-f2bc3bee2f30 None admin] PATH_INFO: `/v3/projects/4de86915ef4340248ba8a8f51d56594e/users/ee422ea90557444fabdc9c292865b9b8/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:32.585615 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282526ae-2d97-4773-90da-f2bc3bee2f30 None admin] RBAC: Authorizing `identity:create_grant(project_id=4de86915ef4340248ba8a8f51d56594e, user_id=ee422ea90557444fabdc9c292865b9b8, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:32.587213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282526ae-2d97-4773-90da-f2bc3bee2f30 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:32.597017 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:32.617216 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 406/815] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 17 23:31:32 2022] PUT /identity/v3/projects/4de86915ef4340248ba8a8f51d56594e/users/ee422ea90557444fabdc9c292865b9b8/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:31:32.688550 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f6498-7133-4125-b855-9d199d287599 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:32.727094 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff868b1e-ebca-4320-adc3-fb5c05a1ec8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:32.727716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff868b1e-ebca-4320-adc3-fb5c05a1ec8d None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:32.728143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff868b1e-ebca-4320-adc3-fb5c05a1ec8d None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:32.728424 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff868b1e-ebca-4320-adc3-fb5c05a1ec8d None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:32.729646 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff868b1e-ebca-4320-adc3-fb5c05a1ec8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:32.729953 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff868b1e-ebca-4320-adc3-fb5c05a1ec8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:32.731327 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff868b1e-ebca-4320-adc3-fb5c05a1ec8d None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:32.740968 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 410/816] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:31:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:32.747347 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282526ae-2d97-4773-90da-f2bc3bee2f30 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:32.843917 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e28f60-89e1-4355-b299-9f5c9adc3a28 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:32.844242 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e28f60-89e1-4355-b299-9f5c9adc3a28 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:32.844426 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e28f60-89e1-4355-b299-9f5c9adc3a28 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:32.846002 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 411/817] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:32.851398 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c28be8-d5a5-4a0c-89ad-680062e849c7 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:32.851686 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c28be8-d5a5-4a0c-89ad-680062e849c7 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:32.851987 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c28be8-d5a5-4a0c-89ad-680062e849c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:32.877245 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-79c28be8-d5a5-4a0c-89ad-680062e849c7 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:32.878465 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-79c28be8-d5a5-4a0c-89ad-680062e849c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:32.878763 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282526ae-2d97-4773-90da-f2bc3bee2f30 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:32.891288 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:32.921469 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c2cba0-6ab3-4e7a-a80a-f70564bfcd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:32.922331 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2cba0-6ab3-4e7a-a80a-f70564bfcd62 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:32.922715 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2cba0-6ab3-4e7a-a80a-f70564bfcd62 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:32.923087 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2cba0-6ab3-4e7a-a80a-f70564bfcd62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:32.935458 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c2cba0-6ab3-4e7a-a80a-f70564bfcd62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:32.937161 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c2cba0-6ab3-4e7a-a80a-f70564bfcd62 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:32.952346 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-43c2cba0-6ab3-4e7a-a80a-f70564bfcd62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:32.952346 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:32.952346 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:32.952346 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:32.952346 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:32.952346 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:32.952346 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:32.952346 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:32.952346 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:32.952346 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:32.952346 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:32.952346 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:32.952346 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:32.952346 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:32.952346 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:32.952346 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:32.953361 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:32.953361 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:32.953361 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:32.953361 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:32.953361 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:32.953361 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:32.953361 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:32.953361 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:32.953361 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:32.953361 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:32.953361 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:32.953361 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:32.953361 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:32.953361 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:32.953361 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:32.953361 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:32.953361 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:32.954274 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:32.954274 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:32.954274 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:32.956712 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 407/818] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:32.959892 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c2cba0-6ab3-4e7a-a80a-f70564bfcd62 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:32.973630 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 412/819] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:31:33.067023 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c2cba0-6ab3-4e7a-a80a-f70564bfcd62 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:33.102412 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c413aa7-0291-4763-a780-97e59a498412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:33.103348 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c413aa7-0291-4763-a780-97e59a498412 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:33.104467 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c413aa7-0291-4763-a780-97e59a498412 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:33.105049 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c413aa7-0291-4763-a780-97e59a498412 None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:33.105896 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c413aa7-0291-4763-a780-97e59a498412 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:33.106423 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c413aa7-0291-4763-a780-97e59a498412 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:33.108016 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c413aa7-0291-4763-a780-97e59a498412 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:33.120893 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 408/820] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:31:33.129067 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c28be8-d5a5-4a0c-89ad-680062e849c7 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:33.160835 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:33.248423 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c28be8-d5a5-4a0c-89ad-680062e849c7 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:33.284910 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed78d4-b4f1-42ae-b7f0-b497080db39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:33.285915 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed78d4-b4f1-42ae-b7f0-b497080db39f None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:33.286399 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed78d4-b4f1-42ae-b7f0-b497080db39f None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:33.286933 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed78d4-b4f1-42ae-b7f0-b497080db39f None admin] PATH_INFO: `/v3/users/nova` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:33.295236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed78d4-b4f1-42ae-b7f0-b497080db39f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:33.296998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed78d4-b4f1-42ae-b7f0-b497080db39f None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:33.309936 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-aeed78d4-b4f1-42ae-b7f0-b497080db39f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 17 23:31:33.309936 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:33.309936 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:33.309936 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:33.309936 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:33.309936 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:33.309936 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:33.309936 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:33.309936 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:33.309936 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:33.309936 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:33.309936 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:33.309936 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 23:31:33.309936 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 23:31:33.309936 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 23:31:33.309936 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 23:31:33.311435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:33.311435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:33.311435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 23:31:33.311435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:33.311435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 23:31:33.311435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:33.311435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:33.311435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:33.311435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:33.311435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:33.311435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:33.311435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:33.311435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:33.311435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:33.311435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:33.311435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:33.311435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:33.311435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:33.312431 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:33.312431 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:33.312431 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 23:31:33.312431 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 23:31:33.312431 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 23:31:33.312431 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 23:31:33.312431 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 23:31:33.312431 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 23:31:33.312431 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 17 23:31:33.312431 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:33.314959 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 413/821] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Aug 17 23:31:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:33.319988 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c413aa7-0291-4763-a780-97e59a498412 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:33.320957 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed78d4-b4f1-42ae-b7f0-b497080db39f None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:33.384808 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:33.417476 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:33.470935 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed78d4-b4f1-42ae-b7f0-b497080db39f None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:33.503197 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c413aa7-0291-4763-a780-97e59a498412 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:33.546803 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c3e0b-c46c-490c-abdc-e274c01de776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:33.547437 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c3e0b-c46c-490c-abdc-e274c01de776 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:33.547715 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c3e0b-c46c-490c-abdc-e274c01de776 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:33.547998 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c3e0b-c46c-490c-abdc-e274c01de776 None admin] PATH_INFO: `/v3/services/network` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:33.548586 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c3e0b-c46c-490c-abdc-e274c01de776 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:33.549953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c3e0b-c46c-490c-abdc-e274c01de776 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:33.552293 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d8c687-5857-4c4b-83fb-1c1dc92e7878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:33.552895 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8c687-5857-4c4b-83fb-1c1dc92e7878 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:33.553172 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8c687-5857-4c4b-83fb-1c1dc92e7878 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:33.553438 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8c687-5857-4c4b-83fb-1c1dc92e7878 None admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:33.554168 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d8c687-5857-4c4b-83fb-1c1dc92e7878 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:33.554473 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d8c687-5857-4c4b-83fb-1c1dc92e7878 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:33.555869 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d8c687-5857-4c4b-83fb-1c1dc92e7878 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:33.574307 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-481c3e0b-c46c-490c-abdc-e274c01de776 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 17 23:31:33.574307 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:33.574307 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 23:31:33.574307 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 23:31:33.574307 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 23:31:33.574307 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 23:31:33.574307 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 23:31:33.574307 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:33.574307 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 17 23:31:33.574307 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:33.574307 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 23:31:33.574307 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:33.574307 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:33.574307 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:33.574307 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:33.574307 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:33.574307 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:33.575334 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:33.575334 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:33.575334 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:33.575334 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:33.575334 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:33.575334 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:33.575334 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 23:31:33.575334 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 23:31:33.575334 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 23:31:33.575334 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 23:31:33.575334 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:33.575334 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:33.575334 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:33.575334 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:33.575334 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:33.575334 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:33.576223 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:33.576223 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:33.576223 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:33.576223 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:33.576223 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:33.576223 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:33.576223 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:33.576223 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:33.576223 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:33.576223 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:33.576223 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 23:31:33.576223 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:33.576223 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 17 23:31:33.576223 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:33.576223 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 409/823] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:31:33 2022] GET /identity/v3/services/network => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:33.579787 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 414/823] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Aug 17 23:31:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:31:33.584905 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c3e0b-c46c-490c-abdc-e274c01de776 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:33.586147 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d8c687-5857-4c4b-83fb-1c1dc92e7878 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:33.688051 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:33.706968 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c3e0b-c46c-490c-abdc-e274c01de776 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:33.756048 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad0ac2-463d-4d13-b364-ecb9f45b2bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:33.757028 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad0ac2-463d-4d13-b364-ecb9f45b2bb9 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:33.757509 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad0ac2-463d-4d13-b364-ecb9f45b2bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:33.757983 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad0ac2-463d-4d13-b364-ecb9f45b2bb9 None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:33.758833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad0ac2-463d-4d13-b364-ecb9f45b2bb9 None admin] RBAC: Adding query filter params (name=network) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:33.759337 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad0ac2-463d-4d13-b364-ecb9f45b2bb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:33.759454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d8c687-5857-4c4b-83fb-1c1dc92e7878 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:33.761093 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad0ac2-463d-4d13-b364-ecb9f45b2bb9 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:33.766407 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 410/824] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Aug 17 23:31:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:33.767833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c15e2a-0d64-43e6-be90-92ea81d5e38b None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:33.768344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c15e2a-0d64-43e6-be90-92ea81d5e38b None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:33.768822 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c15e2a-0d64-43e6-be90-92ea81d5e38b None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:33.774338 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 411/825] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:33.775323 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c15e2a-0d64-43e6-be90-92ea81d5e38b None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:33.794946 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06749217-12fa-42de-a0c9-1b78621e31b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:33.795556 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06749217-12fa-42de-a0c9-1b78621e31b9 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:33.795832 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06749217-12fa-42de-a0c9-1b78621e31b9 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:33.796134 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06749217-12fa-42de-a0c9-1b78621e31b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:33.814475 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:33.865059 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06749217-12fa-42de-a0c9-1b78621e31b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:33.866805 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06749217-12fa-42de-a0c9-1b78621e31b9 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:33.887831 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c15e2a-0d64-43e6-be90-92ea81d5e38b None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:33.923785 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3a87a-9f57-458e-be69-5d42aa240571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:33.924667 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3a87a-9f57-458e-be69-5d42aa240571 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:33.925095 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3a87a-9f57-458e-be69-5d42aa240571 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:33.925510 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3a87a-9f57-458e-be69-5d42aa240571 None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:33.926598 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c3a87a-9f57-458e-be69-5d42aa240571 None admin] RBAC: Adding query filter params (type=network) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:33.927119 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c3a87a-9f57-458e-be69-5d42aa240571 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:33.928729 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c3a87a-9f57-458e-be69-5d42aa240571 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:33.934789 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 412/826] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Aug 17 23:31:33 2022] GET /identity/v3/services?type=network => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:33.936324 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482e854-35d2-406a-97b5-8f605358dfed None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:33.936816 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482e854-35d2-406a-97b5-8f605358dfed None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:33.937337 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482e854-35d2-406a-97b5-8f605358dfed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:33.956208 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-a482e854-35d2-406a-97b5-8f605358dfed None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:33.957713 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-a482e854-35d2-406a-97b5-8f605358dfed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:33.965782 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:33.969403 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 415/827] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:31:33 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 23:31:33.970016 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06749217-12fa-42de-a0c9-1b78621e31b9 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:34.024279 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:34.047266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 413/828] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:31:34.049094 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482e854-35d2-406a-97b5-8f605358dfed None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:34.109951 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06749217-12fa-42de-a0c9-1b78621e31b9 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:34.148580 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4971cae5-a37c-4cef-8dc7-67215f5c9ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:34.149342 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971cae5-a37c-4cef-8dc7-67215f5c9ca4 None admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:34.149628 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971cae5-a37c-4cef-8dc7-67215f5c9ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:34.149908 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971cae5-a37c-4cef-8dc7-67215f5c9ca4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:34.150613 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4971cae5-a37c-4cef-8dc7-67215f5c9ca4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:34.151998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4971cae5-a37c-4cef-8dc7-67215f5c9ca4 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:34.185211 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 416/829] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:31:33 2022] POST /identity/v3/endpoints => generated 337 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:31:34.223091 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482e854-35d2-406a-97b5-8f605358dfed None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:34.266233 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cba014-fea2-434c-a117-68ea3639bfeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:34.266863 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cba014-fea2-434c-a117-68ea3639bfeb None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:34.267138 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cba014-fea2-434c-a117-68ea3639bfeb None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:34.267405 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cba014-fea2-434c-a117-68ea3639bfeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:34.274396 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:34.278853 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cba014-fea2-434c-a117-68ea3639bfeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:34.280378 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cba014-fea2-434c-a117-68ea3639bfeb None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:34.296919 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-75cba014-fea2-434c-a117-68ea3639bfeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:34.296919 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:34.296919 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:34.296919 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:34.296919 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:34.296919 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:34.296919 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:34.296919 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:34.296919 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:34.296919 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:34.296919 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:34.296919 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:34.296919 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:34.296919 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:34.296919 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:34.296919 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:34.297928 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:34.297928 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:34.297928 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:34.297928 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:34.297928 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:34.297928 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:34.297928 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:34.297928 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:34.297928 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:34.297928 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:34.297928 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:34.297928 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:34.297928 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:34.297928 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:34.297928 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:34.297928 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:34.297928 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:34.299121 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:34.299121 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:34.299121 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:34.299121 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 414/830] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:34.305354 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4971cae5-a37c-4cef-8dc7-67215f5c9ca4 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:34.353380 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cba014-fea2-434c-a117-68ea3639bfeb None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:34.422093 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:34.427523 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4971cae5-a37c-4cef-8dc7-67215f5c9ca4 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:34.469823 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c50ab-e6a4-496c-bd38-e57f046e28a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:34.470435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c50ab-e6a4-496c-bd38-e57f046e28a2 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:34.470712 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c50ab-e6a4-496c-bd38-e57f046e28a2 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:34.470980 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c50ab-e6a4-496c-bd38-e57f046e28a2 None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:34.471899 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1c50ab-e6a4-496c-bd38-e57f046e28a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:34.471899 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1c50ab-e6a4-496c-bd38-e57f046e28a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:34.473384 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1c50ab-e6a4-496c-bd38-e57f046e28a2 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:34.488331 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 417/831] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:34.491582 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ccf946-4b2c-47b1-b898-c519ec4fc565 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:34.491582 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ccf946-4b2c-47b1-b898-c519ec4fc565 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:34.491582 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ccf946-4b2c-47b1-b898-c519ec4fc565 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:34.491582 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 418/832] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:34.495247 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ccf946-4b2c-47b1-b898-c519ec4fc565 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:34.505224 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cba014-fea2-434c-a117-68ea3639bfeb None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:34.546936 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd01965-ee2c-4b22-97d3-8656f2ea80d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:34.547561 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd01965-ee2c-4b22-97d3-8656f2ea80d3 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:34.547836 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd01965-ee2c-4b22-97d3-8656f2ea80d3 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:34.548132 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd01965-ee2c-4b22-97d3-8656f2ea80d3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:34.559298 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd01965-ee2c-4b22-97d3-8656f2ea80d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:31:34.565534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-7bd01965-ee2c-4b22-97d3-8656f2ea80d3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:31:34.565534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:34.565534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:34.565534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:34.565534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:34.565534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:34.565534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:34.565534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:34.565534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:34.565534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:34.565534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:34.565534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:34.565534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:31:34.565534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:31:34.565534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:31:34.565534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:31:34.566507 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:31:34.566507 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:31:34.566507 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:31:34.566507 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:31:34.566507 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:34.566507 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:34.566507 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:34.566507 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:34.566507 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:34.566507 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:34.566507 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:34.566507 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:34.566507 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:34.566507 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:34.566507 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:34.566507 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:34.567366 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:34.567366 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:34.567366 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:34.567366 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:34.567366 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:31:34.567366 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:31:34.567366 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:31:34.567366 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:31:34.567366 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:31:34.567366 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:31:34.567366 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:31:34.567366 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:34.567366 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 415/833] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 17 23:31:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:34.568096 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b03ff-c557-43ed-bf41-bc7057e02f91 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:34.568164 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b03ff-c557-43ed-bf41-bc7057e02f91 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:34.568598 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b03ff-c557-43ed-bf41-bc7057e02f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:34.585426 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-c71b03ff-c557-43ed-bf41-bc7057e02f91 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:34.586607 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-c71b03ff-c557-43ed-bf41-bc7057e02f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:34.618705 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ccf946-4b2c-47b1-b898-c519ec4fc565 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:34.642116 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:34.655012 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32954c62-f7d1-48b4-80fc-51c48cb5c662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:34.655645 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32954c62-f7d1-48b4-80fc-51c48cb5c662 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:34.655927 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32954c62-f7d1-48b4-80fc-51c48cb5c662 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:34.656232 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32954c62-f7d1-48b4-80fc-51c48cb5c662 None admin] PATH_INFO: `/v3/projects/service` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:34.663097 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32954c62-f7d1-48b4-80fc-51c48cb5c662 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:34.664582 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32954c62-f7d1-48b4-80fc-51c48cb5c662 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:34.668518 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 416/834] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:34 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:31:34.669252 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71b03ff-c557-43ed-bf41-bc7057e02f91 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:34.688700 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-32954c62-f7d1-48b4-80fc-51c48cb5c662 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:34.688700 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:34.688700 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:34.688700 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:34.688700 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:34.688700 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:34.688700 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:34.688700 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:34.688700 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:34.688700 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:34.688700 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:34.688700 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:34.688700 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 23:31:34.688700 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 23:31:34.688700 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 23:31:34.689652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 23:31:34.689652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:34.689652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:34.689652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:34.689652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:34.689652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:34.689652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:34.689652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:34.689652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:34.689652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:34.689652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:34.689652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:34.689652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:34.689652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:34.689652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:34.689652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:34.689652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:34.690553 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 23:31:34.690553 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 23:31:34.690553 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 23:31:34.690553 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 23:31:34.690553 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 23:31:34.690553 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 23:31:34.690553 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:34.690553 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:34.693405 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 419/835] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:31:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:34.699583 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32954c62-f7d1-48b4-80fc-51c48cb5c662 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:34.724757 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:34.780987 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71b03ff-c557-43ed-bf41-bc7057e02f91 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:34.814207 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32954c62-f7d1-48b4-80fc-51c48cb5c662 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:34.816087 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96851e7a-40c6-49d5-ae53-16936a325e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:34.816688 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96851e7a-40c6-49d5-ae53-16936a325e36 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:34.816963 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96851e7a-40c6-49d5-ae53-16936a325e36 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:34.817230 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96851e7a-40c6-49d5-ae53-16936a325e36 None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:34.818339 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96851e7a-40c6-49d5-ae53-16936a325e36 None admin] RBAC: Adding query filter params (name=service) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:34.818339 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96851e7a-40c6-49d5-ae53-16936a325e36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:34.819528 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96851e7a-40c6-49d5-ae53-16936a325e36 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:34.832944 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 417/836] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 17 23:31:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:34.835024 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96851e7a-40c6-49d5-ae53-16936a325e36 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:34.860302 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc1a86-6e1b-448e-ac3d-267ea0a2d468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:34.860960 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc1a86-6e1b-448e-ac3d-267ea0a2d468 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:34.861514 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc1a86-6e1b-448e-ac3d-267ea0a2d468 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:34.861855 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc1a86-6e1b-448e-ac3d-267ea0a2d468 None admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:34.862546 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dc1a86-6e1b-448e-ac3d-267ea0a2d468 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:34.862852 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dc1a86-6e1b-448e-ac3d-267ea0a2d468 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:34.864354 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dc1a86-6e1b-448e-ac3d-267ea0a2d468 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:34.879999 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 420/837] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 17 23:31:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:34.886986 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc1a86-6e1b-448e-ac3d-267ea0a2d468 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:34.896115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:34.968458 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96851e7a-40c6-49d5-ae53-16936a325e36 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:34.981243 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:35.004080 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc1a86-6e1b-448e-ac3d-267ea0a2d468 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:35.004512 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73dc528-ee0c-48ec-9e87-bfb193f48f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:35.005139 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73dc528-ee0c-48ec-9e87-bfb193f48f28 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:35.005425 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73dc528-ee0c-48ec-9e87-bfb193f48f28 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:35.005699 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73dc528-ee0c-48ec-9e87-bfb193f48f28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:35.016558 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73dc528-ee0c-48ec-9e87-bfb193f48f28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:35.018006 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73dc528-ee0c-48ec-9e87-bfb193f48f28 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:35.032663 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-d73dc528-ee0c-48ec-9e87-bfb193f48f28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:35.032663 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:35.032663 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:35.032663 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:35.032663 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:35.032663 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:35.032663 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:35.032663 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:35.032663 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:35.032663 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:35.032663 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:35.032663 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:35.032663 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:35.032663 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:35.032663 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:35.032663 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:35.033703 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:35.033703 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:35.033703 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:35.033703 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:35.033703 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:35.033703 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:35.033703 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:35.033703 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:35.033703 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:35.033703 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:35.033703 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:35.033703 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:35.033703 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:35.033703 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:35.033703 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:35.033703 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:35.033703 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:35.035234 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:35.035234 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:35.035234 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:35.035234 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 418/838] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:35.035234 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73dc528-ee0c-48ec-9e87-bfb193f48f28 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:35.045579 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc7293-a712-4474-9633-e7e023ec880d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:35.046492 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc7293-a712-4474-9633-e7e023ec880d None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:35.046775 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc7293-a712-4474-9633-e7e023ec880d None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:35.047206 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc7293-a712-4474-9633-e7e023ec880d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:35.047930 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fc7293-a712-4474-9633-e7e023ec880d None admin] RBAC: Adding query filter params (role.id=33bd60d3b0e94dcbbfe97e87f91a8d78, scope.project.id=4de86915ef4340248ba8a8f51d56594e, user.id=ec1dcbe43b0749c3bcc4af9100d771d1) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:35.048271 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fc7293-a712-4474-9633-e7e023ec880d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:35.049691 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fc7293-a712-4474-9633-e7e023ec880d None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:35.055762 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 421/839] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:31:34 2022] GET /identity/v3/role_assignments?user.id=ec1dcbe43b0749c3bcc4af9100d771d1&scope.project.id=4de86915ef4340248ba8a8f51d56594e&role.id=33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:35.057241 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691dc6b-ef58-428c-b9dd-e2c56dcbc3b4 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:35.057526 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691dc6b-ef58-428c-b9dd-e2c56dcbc3b4 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:35.057945 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691dc6b-ef58-428c-b9dd-e2c56dcbc3b4 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:35.059954 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 422/840] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:35.064735 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9691dc6b-ef58-428c-b9dd-e2c56dcbc3b4 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:35.143306 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:35.152333 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73dc528-ee0c-48ec-9e87-bfb193f48f28 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:35.182931 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9691dc6b-ef58-428c-b9dd-e2c56dcbc3b4 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:35.191047 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b605b6b-b9e2-40ad-ac9d-6a3c7b8f76de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:35.191682 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b605b6b-b9e2-40ad-ac9d-6a3c7b8f76de None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:35.191986 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b605b6b-b9e2-40ad-ac9d-6a3c7b8f76de None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:35.192270 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b605b6b-b9e2-40ad-ac9d-6a3c7b8f76de None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:35.192863 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b605b6b-b9e2-40ad-ac9d-6a3c7b8f76de None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:35.194254 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b605b6b-b9e2-40ad-ac9d-6a3c7b8f76de None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:35.204176 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-1b605b6b-b9e2-40ad-ac9d-6a3c7b8f76de None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 17 23:31:35.204176 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:35.204176 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 23:31:35.204176 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 23:31:35.204176 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 23:31:35.204176 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 23:31:35.204176 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 23:31:35.204176 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:35.204176 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 17 23:31:35.204176 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:35.204176 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 23:31:35.204176 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:35.204176 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:35.204176 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:35.204176 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:35.204176 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:35.204176 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:35.205207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:35.205207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:35.205207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:35.205207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:35.205207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:35.205207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:35.205207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 23:31:35.205207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 23:31:35.205207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 23:31:35.205207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 23:31:35.205207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:35.205207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:35.205207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:35.205207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:35.205207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:35.205207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:35.206344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:35.206344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:35.206344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:35.206344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:35.206344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:35.206344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:35.206344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:35.206344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:35.206344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:35.206344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:35.206344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 23:31:35.206344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:35.206344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 17 23:31:35.206344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:35.206344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 419/841] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 17 23:31:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:35.207455 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a52dc-6dac-4952-92e0-d146ea9b3445 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:35.207455 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a52dc-6dac-4952-92e0-d146ea9b3445 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:35.207455 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a52dc-6dac-4952-92e0-d146ea9b3445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:35.227276 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-b23a52dc-6dac-4952-92e0-d146ea9b3445 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:35.227897 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2868b6-6363-476f-9cbb-ad6c961a43ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:35.228613 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-b23a52dc-6dac-4952-92e0-d146ea9b3445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:35.228725 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2868b6-6363-476f-9cbb-ad6c961a43ee None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:35.228914 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2868b6-6363-476f-9cbb-ad6c961a43ee None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:35.229189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2868b6-6363-476f-9cbb-ad6c961a43ee None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:35.229845 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2868b6-6363-476f-9cbb-ad6c961a43ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:35.230149 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2868b6-6363-476f-9cbb-ad6c961a43ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:35.231698 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2868b6-6363-476f-9cbb-ad6c961a43ee None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:35.243213 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 423/842] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:35.244660 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2868b6-6363-476f-9cbb-ad6c961a43ee None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:35.247382 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:35.309797 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 420/843] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:35 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:31:35.312227 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a52dc-6dac-4952-92e0-d146ea9b3445 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:35.364335 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2868b6-6363-476f-9cbb-ad6c961a43ee None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:35.408384 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c41aa39-7937-41b8-8f1b-dbd01b9ee337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:35.409293 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41aa39-7937-41b8-8f1b-dbd01b9ee337 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:35.410031 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41aa39-7937-41b8-8f1b-dbd01b9ee337 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:35.410753 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41aa39-7937-41b8-8f1b-dbd01b9ee337 None admin] PATH_INFO: `/v3/services` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:35.411538 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c41aa39-7937-41b8-8f1b-dbd01b9ee337 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:35.412013 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c41aa39-7937-41b8-8f1b-dbd01b9ee337 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:35.414194 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c41aa39-7937-41b8-8f1b-dbd01b9ee337 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:35.419823 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 424/844] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 17 23:31:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:35.420831 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:35.425529 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c41aa39-7937-41b8-8f1b-dbd01b9ee337 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:35.443147 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a52dc-6dac-4952-92e0-d146ea9b3445 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:35.494167 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77cbe17-150c-4618-b74a-a103f0014d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:35.495198 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77cbe17-150c-4618-b74a-a103f0014d60 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:35.495579 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77cbe17-150c-4618-b74a-a103f0014d60 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:35.495976 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77cbe17-150c-4618-b74a-a103f0014d60 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:35.503545 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77cbe17-150c-4618-b74a-a103f0014d60 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:35.505154 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77cbe17-150c-4618-b74a-a103f0014d60 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:35.510325 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:35.517762 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-e77cbe17-150c-4618-b74a-a103f0014d60 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 17 23:31:35.517762 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:35.517762 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:35.517762 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:35.517762 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:35.517762 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:35.517762 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:35.517762 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:35.517762 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:35.517762 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:35.517762 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:35.517762 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:35.517762 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 23:31:35.517762 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 23:31:35.517762 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 23:31:35.517762 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 23:31:35.518756 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:35.518756 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:35.518756 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 23:31:35.518756 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:35.518756 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 23:31:35.518756 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:35.518756 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:35.518756 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:35.518756 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:35.518756 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:35.518756 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:35.518756 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:35.518756 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:35.518756 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:35.518756 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:35.518756 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:35.518756 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:35.518756 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:35.519701 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:35.519701 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:35.519701 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 23:31:35.519701 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 23:31:35.519701 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 23:31:35.519701 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 23:31:35.519701 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 23:31:35.519701 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 23:31:35.519701 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 17 23:31:35.519701 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:35.523977 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 421/845] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Aug 17 23:31:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:35.525376 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77cbe17-150c-4618-b74a-a103f0014d60 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:35.555360 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c41aa39-7937-41b8-8f1b-dbd01b9ee337 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:35.591867 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdba553-135b-4267-bdf6-1aae3d11df61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:35.592555 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdba553-135b-4267-bdf6-1aae3d11df61 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:35.592992 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdba553-135b-4267-bdf6-1aae3d11df61 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:35.593312 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdba553-135b-4267-bdf6-1aae3d11df61 None admin] PATH_INFO: `/v3/services` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:35.594059 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdba553-135b-4267-bdf6-1aae3d11df61 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:35.594429 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdba553-135b-4267-bdf6-1aae3d11df61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:35.595713 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdba553-135b-4267-bdf6-1aae3d11df61 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:35.604555 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 425/846] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 17 23:31:35 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:35.604555 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdba553-135b-4267-bdf6-1aae3d11df61 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:35.641556 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77cbe17-150c-4618-b74a-a103f0014d60 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:35.691094 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae374156-b781-45c4-a75a-8cc9c1943099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:35.692438 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae374156-b781-45c4-a75a-8cc9c1943099 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:35.693969 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae374156-b781-45c4-a75a-8cc9c1943099 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:35.694442 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae374156-b781-45c4-a75a-8cc9c1943099 None admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:35.695358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae374156-b781-45c4-a75a-8cc9c1943099 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:35.695814 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae374156-b781-45c4-a75a-8cc9c1943099 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:35.697992 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae374156-b781-45c4-a75a-8cc9c1943099 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:35.724586 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 422/847] 10.222.0.26 () {56 vars in 1149 bytes} [Wed Aug 17 23:31:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:35.736560 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae374156-b781-45c4-a75a-8cc9c1943099 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:35.738736 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdba553-135b-4267-bdf6-1aae3d11df61 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:35.747391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:35.777402 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:35.785845 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a719576-559b-4b95-a219-af8aa6b96906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:35.786503 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a719576-559b-4b95-a219-af8aa6b96906 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:35.786778 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a719576-559b-4b95-a219-af8aa6b96906 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:35.787043 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a719576-559b-4b95-a219-af8aa6b96906 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:35.797232 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a719576-559b-4b95-a219-af8aa6b96906 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:31:35.802369 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-5a719576-559b-4b95-a219-af8aa6b96906 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:31:35.802369 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:35.802369 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:35.802369 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:35.802369 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:35.802369 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:35.802369 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:35.802369 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:35.802369 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:35.802369 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:35.802369 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:35.802369 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:35.802369 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:31:35.802369 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:31:35.802369 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:31:35.802369 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:31:35.803334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:31:35.803334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:31:35.803334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:31:35.803334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:31:35.803334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:35.803334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:35.803334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:35.803334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:35.803334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:35.803334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:35.803334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:35.803334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:35.803334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:35.803334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:35.803334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:35.803334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:35.804223 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:35.804223 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:35.804223 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:35.804223 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:35.804223 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:31:35.804223 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:31:35.804223 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:31:35.804223 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:31:35.804223 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:31:35.804223 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:31:35.804223 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:31:35.804223 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:35.804223 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 426/848] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 17 23:31:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:35.810534 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a719576-559b-4b95-a219-af8aa6b96906 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:35.856606 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae374156-b781-45c4-a75a-8cc9c1943099 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:35.892596 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32127d21-58a7-4291-9353-b12d3be80beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:35.893196 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32127d21-58a7-4291-9353-b12d3be80beb None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:35.893472 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32127d21-58a7-4291-9353-b12d3be80beb None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:35.893757 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32127d21-58a7-4291-9353-b12d3be80beb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:35.948577 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a719576-559b-4b95-a219-af8aa6b96906 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:35.968592 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32127d21-58a7-4291-9353-b12d3be80beb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:35.970082 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32127d21-58a7-4291-9353-b12d3be80beb None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:35.978352 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:35.998317 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8be2fc-52f6-47b1-b6e7-a07d04a64b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:35.999436 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8be2fc-52f6-47b1-b6e7-a07d04a64b83 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:36.000193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8be2fc-52f6-47b1-b6e7-a07d04a64b83 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:36.000673 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8be2fc-52f6-47b1-b6e7-a07d04a64b83 None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:36.002157 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8be2fc-52f6-47b1-b6e7-a07d04a64b83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:36.002648 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8be2fc-52f6-47b1-b6e7-a07d04a64b83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:36.004686 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8be2fc-52f6-47b1-b6e7-a07d04a64b83 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:36.018608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 427/849] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:31:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:31:36.026412 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8be2fc-52f6-47b1-b6e7-a07d04a64b83 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:36.089294 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:36.089482 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 423/850] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:31:35 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 23:31:36.090767 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32127d21-58a7-4291-9353-b12d3be80beb None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:36.157129 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8be2fc-52f6-47b1-b6e7-a07d04a64b83 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:36.196838 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32127d21-58a7-4291-9353-b12d3be80beb None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:36.205609 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef92535-f111-4c21-9ca1-4dc0ddfd63de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:36.206249 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef92535-f111-4c21-9ca1-4dc0ddfd63de None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:36.206572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef92535-f111-4c21-9ca1-4dc0ddfd63de None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:36.206842 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef92535-f111-4c21-9ca1-4dc0ddfd63de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:36.207244 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:36.218220 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef92535-f111-4c21-9ca1-4dc0ddfd63de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:36.219672 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef92535-f111-4c21-9ca1-4dc0ddfd63de None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:36.235291 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-aef92535-f111-4c21-9ca1-4dc0ddfd63de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:36.235291 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:36.235291 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:36.235291 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:36.235291 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:36.235291 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:36.235291 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:36.235291 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:36.235291 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:36.235291 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:36.235291 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:36.235291 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:36.235291 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:36.235291 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:36.235291 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:36.235291 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:36.236468 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:36.236468 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:36.236468 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:36.236468 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:36.236468 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:36.236468 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:36.236468 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:36.236468 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:36.236468 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:36.236468 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:36.236468 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:36.236468 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:36.236468 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:36.236468 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:36.236468 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:36.236468 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:36.236468 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:36.237394 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:36.237394 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:36.237394 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:36.240216 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 428/851] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:31:36.242559 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef92535-f111-4c21-9ca1-4dc0ddfd63de None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:36.246945 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7a8ac-7834-429f-bcd3-505318a678be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:36.247570 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7a8ac-7834-429f-bcd3-505318a678be None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:36.248800 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7a8ac-7834-429f-bcd3-505318a678be None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:36.249041 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7a8ac-7834-429f-bcd3-505318a678be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:36.262206 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c7a8ac-7834-429f-bcd3-505318a678be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:36.264076 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c7a8ac-7834-429f-bcd3-505318a678be None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:36.284308 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-a5c7a8ac-7834-429f-bcd3-505318a678be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:36.284308 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:36.284308 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:36.284308 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:36.284308 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:36.284308 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:36.284308 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:36.284308 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:36.284308 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:36.284308 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:36.284308 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:36.284308 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:36.284308 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:36.284308 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:36.284308 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:36.284308 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:36.285298 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:36.285298 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:36.285298 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:36.285298 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:36.285298 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:36.285298 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:36.285298 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:36.285298 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:36.285298 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:36.285298 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:36.285298 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:36.285298 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:36.285298 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:36.285298 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:36.285298 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:36.285298 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:36.285298 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:36.291475 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:36.291475 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:36.291475 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:36.291475 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 424/852] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:36.293136 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7a8ac-7834-429f-bcd3-505318a678be None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:36.357985 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef92535-f111-4c21-9ca1-4dc0ddfd63de None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:36.361407 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:36.395240 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b584d84e-f051-4460-be2d-56f055c69945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:36.396246 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584d84e-f051-4460-be2d-56f055c69945 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:36.396727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584d84e-f051-4460-be2d-56f055c69945 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:36.397244 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584d84e-f051-4460-be2d-56f055c69945 None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:36.398092 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b584d84e-f051-4460-be2d-56f055c69945 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:36.398610 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b584d84e-f051-4460-be2d-56f055c69945 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:36.400199 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b584d84e-f051-4460-be2d-56f055c69945 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:36.411085 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7a8ac-7834-429f-bcd3-505318a678be None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:36.419066 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 429/853] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:31:36.421161 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b584d84e-f051-4460-be2d-56f055c69945 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:36.457818 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d5ac8-01e2-4a5b-b2f6-4feb7ff65b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:36.458489 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d5ac8-01e2-4a5b-b2f6-4feb7ff65b0f None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:36.458765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d5ac8-01e2-4a5b-b2f6-4feb7ff65b0f None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:36.459028 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d5ac8-01e2-4a5b-b2f6-4feb7ff65b0f None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:36.459978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0d5ac8-01e2-4a5b-b2f6-4feb7ff65b0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:36.460304 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0d5ac8-01e2-4a5b-b2f6-4feb7ff65b0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:36.462355 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0d5ac8-01e2-4a5b-b2f6-4feb7ff65b0f None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:36.477893 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 425/854] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:31:36.483954 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d5ac8-01e2-4a5b-b2f6-4feb7ff65b0f None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:36.506342 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:36.543495 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b584d84e-f051-4460-be2d-56f055c69945 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:36.579547 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a1ec1-589a-4d35-a61b-f3cba2bba6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:36.580605 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a1ec1-589a-4d35-a61b-f3cba2bba6f4 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:36.580605 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a1ec1-589a-4d35-a61b-f3cba2bba6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:36.580862 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a1ec1-589a-4d35-a61b-f3cba2bba6f4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:36.588366 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a1ec1-589a-4d35-a61b-f3cba2bba6f4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:36.589802 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a1ec1-589a-4d35-a61b-f3cba2bba6f4 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:36.593947 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d5ac8-01e2-4a5b-b2f6-4feb7ff65b0f None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:36.601769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-bb7a1ec1-589a-4d35-a61b-f3cba2bba6f4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 17 23:31:36.601769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:36.601769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:36.601769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:36.601769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:36.601769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:36.601769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:36.601769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:36.601769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:36.601769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:36.601769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:36.601769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:36.601769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 23:31:36.601769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 23:31:36.601769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 23:31:36.601769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 23:31:36.602735 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:36.602735 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:36.602735 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 23:31:36.602735 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:36.602735 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 23:31:36.602735 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:36.602735 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:36.602735 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:36.602735 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:36.602735 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:36.602735 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:36.602735 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:36.602735 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:36.602735 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:36.602735 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:36.602735 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:36.602735 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:36.602735 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:36.604299 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:36.604299 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:36.604299 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 23:31:36.604299 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 23:31:36.604299 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 23:31:36.604299 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 23:31:36.604299 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 23:31:36.604299 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 23:31:36.604299 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 17 23:31:36.604299 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:36.608170 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 430/855] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 17 23:31:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:36.609564 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a1ec1-589a-4d35-a61b-f3cba2bba6f4 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:36.634819 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7bc007-ea3f-4761-9b68-361653940b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:36.635859 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bc007-ea3f-4761-9b68-361653940b0c None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:36.636359 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bc007-ea3f-4761-9b68-361653940b0c None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:36.636827 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bc007-ea3f-4761-9b68-361653940b0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:36.641433 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:36.644512 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7bc007-ea3f-4761-9b68-361653940b0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:36.646241 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7bc007-ea3f-4761-9b68-361653940b0c None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:36.659723 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-0a7bc007-ea3f-4761-9b68-361653940b0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:36.659723 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:36.659723 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:36.659723 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:36.659723 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:36.659723 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:36.659723 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:36.659723 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:36.659723 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:36.659723 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:36.659723 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:36.659723 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:36.659723 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 23:31:36.659723 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 23:31:36.659723 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 23:31:36.660692 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 23:31:36.660692 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:36.660692 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:36.660692 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:36.660692 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:36.660692 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:36.660692 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:36.660692 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:36.660692 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:36.660692 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:36.660692 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:36.660692 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:36.660692 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:36.660692 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:36.660692 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:36.660692 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:36.660692 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:36.661580 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 23:31:36.661580 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 23:31:36.661580 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 23:31:36.661580 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 23:31:36.661580 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 23:31:36.661580 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 23:31:36.661580 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:36.661580 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:36.668261 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 426/856] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:31:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:31:36.670454 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7bc007-ea3f-4761-9b68-361653940b0c None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:36.723574 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a1ec1-589a-4d35-a61b-f3cba2bba6f4 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:36.755388 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:36.761888 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9160978d-14a7-413f-8189-df0d1970b5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:36.762440 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160978d-14a7-413f-8189-df0d1970b5d4 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:36.762723 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160978d-14a7-413f-8189-df0d1970b5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:36.763002 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160978d-14a7-413f-8189-df0d1970b5d4 None admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:36.763757 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9160978d-14a7-413f-8189-df0d1970b5d4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:36.764112 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9160978d-14a7-413f-8189-df0d1970b5d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:36.765950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9160978d-14a7-413f-8189-df0d1970b5d4 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:36.782247 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7bc007-ea3f-4761-9b68-361653940b0c None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:36.786909 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 431/857] 10.222.0.26 () {56 vars in 1135 bytes} [Wed Aug 17 23:31:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:36.801686 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9160978d-14a7-413f-8189-df0d1970b5d4 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:36.820167 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107598a5-7bd0-43bf-a159-e8afadfe75e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:36.820819 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107598a5-7bd0-43bf-a159-e8afadfe75e9 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:36.821090 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107598a5-7bd0-43bf-a159-e8afadfe75e9 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:36.821385 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107598a5-7bd0-43bf-a159-e8afadfe75e9 None admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:36.822627 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107598a5-7bd0-43bf-a159-e8afadfe75e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:36.823027 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107598a5-7bd0-43bf-a159-e8afadfe75e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:36.824696 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107598a5-7bd0-43bf-a159-e8afadfe75e9 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:36.844117 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 427/858] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 17 23:31:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:36.845924 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107598a5-7bd0-43bf-a159-e8afadfe75e9 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:36.912517 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:36.938038 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9160978d-14a7-413f-8189-df0d1970b5d4 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:36.977511 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107598a5-7bd0-43bf-a159-e8afadfe75e9 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:36.981266 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b097939-1d99-456f-92a6-a34408acc0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:36.982102 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b097939-1d99-456f-92a6-a34408acc0f1 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:36.982603 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b097939-1d99-456f-92a6-a34408acc0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:36.983066 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b097939-1d99-456f-92a6-a34408acc0f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:37.023173 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612930b3-9b87-4d7e-9a83-cb49f9cc28e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:37.023695 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612930b3-9b87-4d7e-9a83-cb49f9cc28e5 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:37.024003 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612930b3-9b87-4d7e-9a83-cb49f9cc28e5 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:37.024266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612930b3-9b87-4d7e-9a83-cb49f9cc28e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:37.024986 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612930b3-9b87-4d7e-9a83-cb49f9cc28e5 None admin] RBAC: Adding query filter params (role.id=b64f80e6f3ea4ad49b94acdde42ccd87, scope.project.id=4de86915ef4340248ba8a8f51d56594e, user.id=d0a13dbffb4a49508b7fb661cf84e803) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:37.025344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612930b3-9b87-4d7e-9a83-cb49f9cc28e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:37.028223 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612930b3-9b87-4d7e-9a83-cb49f9cc28e5 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:37.035590 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 428/859] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:31:36 2022] GET /identity/v3/role_assignments?user.id=d0a13dbffb4a49508b7fb661cf84e803&scope.project.id=4de86915ef4340248ba8a8f51d56594e&role.id=b64f80e6f3ea4ad49b94acdde42ccd87 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:31:37.036878 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69574933-9c3a-457c-8ade-ac8b90d6dae9 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:37.037164 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69574933-9c3a-457c-8ade-ac8b90d6dae9 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:37.037434 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69574933-9c3a-457c-8ade-ac8b90d6dae9 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:37.039133 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 429/860] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:37.045794 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b403e7f-674d-465d-bb49-8ac33be24ee0 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:37.046072 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b403e7f-674d-465d-bb49-8ac33be24ee0 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:37.046878 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b403e7f-674d-465d-bb49-8ac33be24ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:37.057846 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:37.061083 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b097939-1d99-456f-92a6-a34408acc0f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:37.062615 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b097939-1d99-456f-92a6-a34408acc0f1 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:37.071000 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-5b403e7f-674d-465d-bb49-8ac33be24ee0 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:37.073576 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-5b403e7f-674d-465d-bb49-8ac33be24ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:37.154679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:37.176339 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 430/861] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:37 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:31:37.184483 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 432/862] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:31:36 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 23:31:37.190279 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b403e7f-674d-465d-bb49-8ac33be24ee0 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:37.297205 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b403e7f-674d-465d-bb49-8ac33be24ee0 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:37.329679 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:37.338858 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809eb81-fc63-42d9-b80f-88dd1e59afd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:37.340267 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809eb81-fc63-42d9-b80f-88dd1e59afd8 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:37.340617 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809eb81-fc63-42d9-b80f-88dd1e59afd8 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:37.341106 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809eb81-fc63-42d9-b80f-88dd1e59afd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:37.353029 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f809eb81-fc63-42d9-b80f-88dd1e59afd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:37.354792 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f809eb81-fc63-42d9-b80f-88dd1e59afd8 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:37.370595 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-f809eb81-fc63-42d9-b80f-88dd1e59afd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:37.370595 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:37.370595 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:37.370595 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:37.370595 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:37.370595 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:37.370595 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:37.370595 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:37.370595 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:37.370595 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:37.370595 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:37.370595 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:37.370595 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:37.370595 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:37.370595 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:37.370595 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:37.371554 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:37.371554 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:37.371554 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:37.371554 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:37.371554 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:37.371554 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:37.371554 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:37.371554 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:37.371554 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:37.371554 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:37.371554 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:37.371554 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:37.371554 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:37.371554 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:37.371554 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:37.371554 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:37.371554 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:37.372503 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:37.372503 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:37.372503 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:37.374538 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 431/863] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:37.385408 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b097939-1d99-456f-92a6-a34408acc0f1 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:37.460956 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809eb81-fc63-42d9-b80f-88dd1e59afd8 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:37.490211 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b097939-1d99-456f-92a6-a34408acc0f1 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:37.528872 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf2944-f411-4d68-b020-00fdcbff41ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:37.529482 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf2944-f411-4d68-b020-00fdcbff41ee None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:37.529760 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf2944-f411-4d68-b020-00fdcbff41ee None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:37.530035 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf2944-f411-4d68-b020-00fdcbff41ee None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:37.530688 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf2944-f411-4d68-b020-00fdcbff41ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:37.530989 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf2944-f411-4d68-b020-00fdcbff41ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:37.532641 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf2944-f411-4d68-b020-00fdcbff41ee None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:37.549540 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 433/864] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:37.550897 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf2944-f411-4d68-b020-00fdcbff41ee None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:37.568957 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809eb81-fc63-42d9-b80f-88dd1e59afd8 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:37.601578 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:37.605907 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f8d928-d274-4cdc-9a23-dde66496ca46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:37.606539 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f8d928-d274-4cdc-9a23-dde66496ca46 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:37.606816 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f8d928-d274-4cdc-9a23-dde66496ca46 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:37.607084 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f8d928-d274-4cdc-9a23-dde66496ca46 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:37.607671 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f8d928-d274-4cdc-9a23-dde66496ca46 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:37.609212 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f8d928-d274-4cdc-9a23-dde66496ca46 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:37.618846 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-97f8d928-d274-4cdc-9a23-dde66496ca46 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 17 23:31:37.618846 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:37.618846 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 23:31:37.618846 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 23:31:37.618846 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 23:31:37.618846 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 23:31:37.618846 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 23:31:37.618846 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:37.618846 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 17 23:31:37.618846 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:37.618846 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 23:31:37.618846 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:37.618846 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:37.618846 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:37.618846 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:37.618846 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:37.620555 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:37.620555 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:37.620555 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:37.620555 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:37.620555 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:37.620555 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:37.620555 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:37.620555 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 23:31:37.620555 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 23:31:37.620555 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 23:31:37.620555 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 23:31:37.620555 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:37.620555 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:37.620555 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:37.620555 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:37.620555 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:37.620555 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:37.621441 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:37.621441 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:37.621441 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:37.621441 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:37.621441 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:37.621441 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:37.621441 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:37.621441 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:37.621441 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:37.621441 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:37.621441 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 23:31:37.621441 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:37.621441 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 17 23:31:37.621441 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:37.621441 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 432/865] 10.222.0.26 () {56 vars in 1132 bytes} [Wed Aug 17 23:31:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:31:37.622245 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98d4bc-f22f-4ee7-857d-a44fc96b57bb None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:37.622245 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98d4bc-f22f-4ee7-857d-a44fc96b57bb None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:37.622365 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98d4bc-f22f-4ee7-857d-a44fc96b57bb None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:37.623994 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 433/866] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:37.628605 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba8cb1-72e4-4834-961c-fa7f3dd12493 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:37.628893 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba8cb1-72e4-4834-961c-fa7f3dd12493 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:37.629167 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba8cb1-72e4-4834-961c-fa7f3dd12493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:37.647344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-79ba8cb1-72e4-4834-961c-fa7f3dd12493 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:37.648570 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-79ba8cb1-72e4-4834-961c-fa7f3dd12493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:37.665188 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf2944-f411-4d68-b020-00fdcbff41ee None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:37.699868 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:37.703892 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b4a031-e2bf-4f72-8340-037e1c372aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:37.704547 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b4a031-e2bf-4f72-8340-037e1c372aa1 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:37.705189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b4a031-e2bf-4f72-8340-037e1c372aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:37.705189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b4a031-e2bf-4f72-8340-037e1c372aa1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:37.713049 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b4a031-e2bf-4f72-8340-037e1c372aa1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:37.714730 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b4a031-e2bf-4f72-8340-037e1c372aa1 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:37.732664 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-75b4a031-e2bf-4f72-8340-037e1c372aa1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:37.732664 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:37.732664 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:37.732664 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:37.732664 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:37.732664 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:37.732664 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:37.732664 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:37.732664 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:37.732664 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:37.732664 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:37.732664 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:37.732664 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 23:31:37.732664 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 23:31:37.732664 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 23:31:37.733798 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 23:31:37.733798 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:37.733798 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:37.733798 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:37.733798 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:37.733798 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:37.733798 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:37.733798 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:37.733798 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:37.733798 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:37.733798 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:37.733798 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:37.733798 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:37.733798 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:37.733798 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:37.733798 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:37.733798 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:37.734907 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 23:31:37.734907 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 23:31:37.734907 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 23:31:37.734907 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 23:31:37.734907 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 23:31:37.734907 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 23:31:37.734907 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:37.734907 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:37.735441 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 434/867] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:37 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 23:31:37.735441 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba8cb1-72e4-4834-961c-fa7f3dd12493 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:37.735612 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 434/868] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:31:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:37.735612 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b4a031-e2bf-4f72-8340-037e1c372aa1 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:37.846637 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba8cb1-72e4-4834-961c-fa7f3dd12493 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:37.857669 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b4a031-e2bf-4f72-8340-037e1c372aa1 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:37.859983 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:37.885651 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f078b6e1-b253-4946-b184-568aaacd7336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:37.886465 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078b6e1-b253-4946-b184-568aaacd7336 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:37.886780 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078b6e1-b253-4946-b184-568aaacd7336 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:37.887117 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078b6e1-b253-4946-b184-568aaacd7336 None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:37.887816 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f078b6e1-b253-4946-b184-568aaacd7336 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:37.888301 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f078b6e1-b253-4946-b184-568aaacd7336 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:37.889897 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f078b6e1-b253-4946-b184-568aaacd7336 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:37.899449 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 435/869] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Aug 17 23:31:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:37.900897 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f078b6e1-b253-4946-b184-568aaacd7336 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:37.901116 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628c140f-ca4a-4767-9152-480c9b00bd41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:37.901944 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c140f-ca4a-4767-9152-480c9b00bd41 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:37.902389 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c140f-ca4a-4767-9152-480c9b00bd41 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:37.903064 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c140f-ca4a-4767-9152-480c9b00bd41 None admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:37.904104 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628c140f-ca4a-4767-9152-480c9b00bd41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:37.904558 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628c140f-ca4a-4767-9152-480c9b00bd41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:37.906940 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628c140f-ca4a-4767-9152-480c9b00bd41 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:37.922933 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 435/870] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 17 23:31:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:31:37.928595 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628c140f-ca4a-4767-9152-480c9b00bd41 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:37.949690 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:38.014054 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f078b6e1-b253-4946-b184-568aaacd7336 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:38.049546 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628c140f-ca4a-4767-9152-480c9b00bd41 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:38.050028 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842c1d83-5273-4097-acbe-9de584618e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:38.050638 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c1d83-5273-4097-acbe-9de584618e1d None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:38.050927 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c1d83-5273-4097-acbe-9de584618e1d None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:38.051196 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c1d83-5273-4097-acbe-9de584618e1d None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:38.051832 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842c1d83-5273-4097-acbe-9de584618e1d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:38.052198 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842c1d83-5273-4097-acbe-9de584618e1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:38.053542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842c1d83-5273-4097-acbe-9de584618e1d None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:38.067996 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 436/871] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Aug 17 23:31:37 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:38.067996 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842c1d83-5273-4097-acbe-9de584618e1d None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:38.098853 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ed12a-7c8f-46b2-92d0-fbc6abe29c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:38.099836 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ed12a-7c8f-46b2-92d0-fbc6abe29c48 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:38.100407 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ed12a-7c8f-46b2-92d0-fbc6abe29c48 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:38.100683 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ed12a-7c8f-46b2-92d0-fbc6abe29c48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:38.101453 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ed12a-7c8f-46b2-92d0-fbc6abe29c48 None admin] RBAC: Adding query filter params (role.id=33bd60d3b0e94dcbbfe97e87f91a8d78, scope.project.id=4de86915ef4340248ba8a8f51d56594e, user.id=ee422ea90557444fabdc9c292865b9b8) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:38.101987 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ed12a-7c8f-46b2-92d0-fbc6abe29c48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:38.104552 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ed12a-7c8f-46b2-92d0-fbc6abe29c48 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:38.113287 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 436/872] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:31:37 2022] GET /identity/v3/role_assignments?user.id=ee422ea90557444fabdc9c292865b9b8&scope.project.id=4de86915ef4340248ba8a8f51d56594e&role.id=33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:38.188240 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842c1d83-5273-4097-acbe-9de584618e1d None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:38.200417 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:38.230683 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d73fbe-46da-442c-afcb-67ba50b562f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:38.231317 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d73fbe-46da-442c-afcb-67ba50b562f3 None admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:38.231601 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d73fbe-46da-442c-afcb-67ba50b562f3 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:38.231877 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d73fbe-46da-442c-afcb-67ba50b562f3 None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:38.232583 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d73fbe-46da-442c-afcb-67ba50b562f3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:38.234010 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d73fbe-46da-442c-afcb-67ba50b562f3 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:38.254422 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 437/873] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Aug 17 23:31:38 2022] POST /identity/v3/services => generated 251 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:31:39.121373 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7bf69-32a4-482f-9db2-0be51aeb9d00 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:39.121918 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7bf69-32a4-482f-9db2-0be51aeb9d00 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:39.121918 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7bf69-32a4-482f-9db2-0be51aeb9d00 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:39.123477 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 437/874] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:39.128776 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6685f080-9f25-40b5-acc1-4b9f815f6963 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:39.129162 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6685f080-9f25-40b5-acc1-4b9f815f6963 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:39.129324 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6685f080-9f25-40b5-acc1-4b9f815f6963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:39.147454 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-6685f080-9f25-40b5-acc1-4b9f815f6963 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:39.148682 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-6685f080-9f25-40b5-acc1-4b9f815f6963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:39.227633 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 438/875] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:39 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:31:39.502365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d7bf69-32a4-482f-9db2-0be51aeb9d00 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:39.530946 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:39.609523 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d7bf69-32a4-482f-9db2-0be51aeb9d00 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:39.644967 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca2dfd-c022-48a8-8cb1-0addbbffbfc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:39.646022 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca2dfd-c022-48a8-8cb1-0addbbffbfc5 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:39.646510 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca2dfd-c022-48a8-8cb1-0addbbffbfc5 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:39.646997 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca2dfd-c022-48a8-8cb1-0addbbffbfc5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:39.658576 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ca2dfd-c022-48a8-8cb1-0addbbffbfc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:31:39.663572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-39ca2dfd-c022-48a8-8cb1-0addbbffbfc5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 17 23:31:39.663572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:39.663572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:39.663572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:39.663572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:39.663572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:39.663572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:39.663572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:39.663572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:39.663572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:39.663572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:39.663572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:39.663572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:31:39.663572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:31:39.663572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:31:39.663572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:31:39.664501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:31:39.664501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:31:39.664501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:31:39.664501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:31:39.664501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:39.664501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:39.664501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:39.664501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:39.664501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:39.664501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:39.664501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:39.664501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:39.664501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:39.664501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:39.664501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:39.664501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:39.665337 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:39.665337 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:39.665337 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:39.665337 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:39.665337 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:31:39.665337 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:31:39.665337 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:31:39.665337 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:31:39.665337 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:31:39.665337 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:31:39.665337 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 17 23:31:39.665337 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:39.672061 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 438/876] 10.222.0.26 () {56 vars in 1120 bytes} [Wed Aug 17 23:31:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:39.677115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6685f080-9f25-40b5-acc1-4b9f815f6963 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:39.769044 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:39.781382 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6685f080-9f25-40b5-acc1-4b9f815f6963 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:39.817453 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399a8428-afd7-47ed-9850-ca717e4945a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:39.818462 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a8428-afd7-47ed-9850-ca717e4945a0 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:39.818971 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a8428-afd7-47ed-9850-ca717e4945a0 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:39.819466 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a8428-afd7-47ed-9850-ca717e4945a0 None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:39.820392 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399a8428-afd7-47ed-9850-ca717e4945a0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:39.820901 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399a8428-afd7-47ed-9850-ca717e4945a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:39.822475 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399a8428-afd7-47ed-9850-ca717e4945a0 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:39.835880 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 439/877] 10.222.0.26 () {56 vars in 1115 bytes} [Wed Aug 17 23:31:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:39.837538 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca2dfd-c022-48a8-8cb1-0addbbffbfc5 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:39.927992 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:39.943905 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca2dfd-c022-48a8-8cb1-0addbbffbfc5 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:39.980110 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516cf4a3-3998-4b0b-8dc2-a006d019d74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:39.981128 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cf4a3-3998-4b0b-8dc2-a006d019d74b None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:39.981613 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cf4a3-3998-4b0b-8dc2-a006d019d74b None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:39.982115 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cf4a3-3998-4b0b-8dc2-a006d019d74b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:39.992759 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516cf4a3-3998-4b0b-8dc2-a006d019d74b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:39.994458 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516cf4a3-3998-4b0b-8dc2-a006d019d74b None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:40.010149 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-516cf4a3-3998-4b0b-8dc2-a006d019d74b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:40.010149 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:40.010149 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:40.010149 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:40.010149 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:40.010149 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:40.010149 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:40.010149 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:40.010149 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:40.010149 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:40.010149 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:40.010149 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:40.010149 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:40.010149 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:40.010149 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:40.010149 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:40.011150 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:40.011150 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:40.011150 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:40.011150 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:40.011150 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:40.011150 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:40.011150 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:40.011150 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:40.011150 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:40.011150 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:40.011150 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:40.011150 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:40.011150 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:40.011150 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:40.011150 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:40.011150 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:40.011150 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:40.012079 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:40.012079 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:40.012079 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:40.018689 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 439/878] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:40.020409 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399a8428-afd7-47ed-9850-ca717e4945a0 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:40.071432 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05fb015-04be-4dd8-a9d0-31dd2b33cdf5 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:40.072624 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05fb015-04be-4dd8-a9d0-31dd2b33cdf5 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:40.073117 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05fb015-04be-4dd8-a9d0-31dd2b33cdf5 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:40.075003 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 440/879] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 23:31:40.081273 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ff4cb-b2a1-479d-997b-7f2232b4e05f None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:40.081690 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ff4cb-b2a1-479d-997b-7f2232b4e05f None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:40.082056 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ff4cb-b2a1-479d-997b-7f2232b4e05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:40.101942 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-a36ff4cb-b2a1-479d-997b-7f2232b4e05f None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:40.103429 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-a36ff4cb-b2a1-479d-997b-7f2232b4e05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:40.128005 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399a8428-afd7-47ed-9850-ca717e4945a0 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:40.166828 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278ea6c-5b6e-44e6-a95b-dc3d93d23e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:40.167596 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278ea6c-5b6e-44e6-a95b-dc3d93d23e3d None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:40.167880 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278ea6c-5b6e-44e6-a95b-dc3d93d23e3d None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:40.168241 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278ea6c-5b6e-44e6-a95b-dc3d93d23e3d None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:40.169036 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d278ea6c-5b6e-44e6-a95b-dc3d93d23e3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:40.169340 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d278ea6c-5b6e-44e6-a95b-dc3d93d23e3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:40.170711 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d278ea6c-5b6e-44e6-a95b-dc3d93d23e3d None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:40.177384 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 441/880] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:40 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:31:40.178470 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c2445-3a1c-4dca-ae1d-43cdc16c7482 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:40.178758 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c2445-3a1c-4dca-ae1d-43cdc16c7482 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:40.179034 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c2445-3a1c-4dca-ae1d-43cdc16c7482 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:40.180424 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 442/881] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:40.185679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11dd68-5261-4f16-80f2-86fe40ac007d None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:40.185968 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11dd68-5261-4f16-80f2-86fe40ac007d None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:40.186561 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11dd68-5261-4f16-80f2-86fe40ac007d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:40.191695 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 440/882] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:40.198545 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278ea6c-5b6e-44e6-a95b-dc3d93d23e3d None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:40.210679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-0c11dd68-5261-4f16-80f2-86fe40ac007d None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:40.211939 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-0c11dd68-5261-4f16-80f2-86fe40ac007d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:40.215956 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:40.279631 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:40.286694 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 443/883] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:40 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 23:31:40.287936 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18545504-001e-4caa-aef6-6386bc953d5b None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:40.288250 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18545504-001e-4caa-aef6-6386bc953d5b None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:40.288525 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18545504-001e-4caa-aef6-6386bc953d5b None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:40.290023 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 444/884] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:40.295562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a2d32-d204-40d8-b51e-3caba18a117d None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:40.295928 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a2d32-d204-40d8-b51e-3caba18a117d None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:40.296494 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a2d32-d204-40d8-b51e-3caba18a117d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:40.314324 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278ea6c-5b6e-44e6-a95b-dc3d93d23e3d None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:40.319284 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-275a2d32-d204-40d8-b51e-3caba18a117d None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:40.320497 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-275a2d32-d204-40d8-b51e-3caba18a117d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:40.350970 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36feee75-90fc-4790-b6d5-89200ae311cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:40.351729 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36feee75-90fc-4790-b6d5-89200ae311cf None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:40.352172 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36feee75-90fc-4790-b6d5-89200ae311cf None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:40.352570 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36feee75-90fc-4790-b6d5-89200ae311cf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:40.361715 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36feee75-90fc-4790-b6d5-89200ae311cf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:40.363187 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36feee75-90fc-4790-b6d5-89200ae311cf None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:40.378141 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-36feee75-90fc-4790-b6d5-89200ae311cf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 17 23:31:40.378141 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:40.378141 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:40.378141 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:40.378141 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:40.378141 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:40.378141 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:40.378141 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:40.378141 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:40.378141 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:40.378141 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:40.378141 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:40.378141 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 23:31:40.378141 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 23:31:40.378141 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 23:31:40.378141 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 23:31:40.379104 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:40.379104 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:40.379104 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 23:31:40.379104 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:40.379104 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 23:31:40.379104 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:40.379104 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:40.379104 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:40.379104 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:40.379104 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:40.379104 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:40.379104 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:40.379104 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:40.379104 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:40.379104 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:40.379104 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:40.379104 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:40.379104 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:40.380094 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:40.380094 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:40.380094 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 23:31:40.380094 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 23:31:40.380094 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 23:31:40.380094 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 23:31:40.380094 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 23:31:40.380094 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 23:31:40.380094 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 17 23:31:40.380094 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:40.384315 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 441/885] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Aug 17 23:31:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:31:40.385136 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36feee75-90fc-4790-b6d5-89200ae311cf None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:40.416492 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 445/886] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:40 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 23:31:40.462185 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275a2d32-d204-40d8-b51e-3caba18a117d None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:40.481316 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:40.508821 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36feee75-90fc-4790-b6d5-89200ae311cf None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:40.551246 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ebe66-3110-4cf0-afbb-850457b7df4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:40.551776 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ebe66-3110-4cf0-afbb-850457b7df4f None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:40.552097 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ebe66-3110-4cf0-afbb-850457b7df4f None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:40.552438 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ebe66-3110-4cf0-afbb-850457b7df4f None admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:40.553284 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825ebe66-3110-4cf0-afbb-850457b7df4f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:40.553642 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825ebe66-3110-4cf0-afbb-850457b7df4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:40.555301 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825ebe66-3110-4cf0-afbb-850457b7df4f None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:40.572719 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 442/887] 10.222.0.26 () {56 vars in 1149 bytes} [Wed Aug 17 23:31:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:31:40.576324 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ebe66-3110-4cf0-afbb-850457b7df4f None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:40.587533 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275a2d32-d204-40d8-b51e-3caba18a117d None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:40.616696 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:40.625880 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ec309-ba40-4f89-b037-e111c5b2b56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:40.626600 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ec309-ba40-4f89-b037-e111c5b2b56a None admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:40.626883 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ec309-ba40-4f89-b037-e111c5b2b56a None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:40.627211 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ec309-ba40-4f89-b037-e111c5b2b56a None admin] PATH_INFO: `/v3/services` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:40.627870 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1ec309-ba40-4f89-b037-e111c5b2b56a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:40.629432 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1ec309-ba40-4f89-b037-e111c5b2b56a None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:40.646131 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 446/888] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Aug 17 23:31:40 2022] POST /identity/v3/services => generated 269 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:31:40.647031 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ec309-ba40-4f89-b037-e111c5b2b56a None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:40.700011 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ebe66-3110-4cf0-afbb-850457b7df4f None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:40.743126 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:40.754853 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183d0e9f-0747-4885-a390-4938e8c664c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:40.755503 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d0e9f-0747-4885-a390-4938e8c664c3 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:40.755767 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d0e9f-0747-4885-a390-4938e8c664c3 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:40.756064 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d0e9f-0747-4885-a390-4938e8c664c3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:40.756657 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183d0e9f-0747-4885-a390-4938e8c664c3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:40.758058 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183d0e9f-0747-4885-a390-4938e8c664c3 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:40.768133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ec309-ba40-4f89-b037-e111c5b2b56a None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:40.768402 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-183d0e9f-0747-4885-a390-4938e8c664c3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 23:31:40.768402 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:40.768402 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 23:31:40.768402 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 23:31:40.768402 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 23:31:40.768402 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 23:31:40.768402 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 23:31:40.768402 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:40.768402 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 23:31:40.768402 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:40.768402 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 23:31:40.768402 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:40.768402 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:40.768402 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:40.768402 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:40.768402 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:40.768402 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:40.769394 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:40.769394 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:40.769394 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:40.769394 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:40.769394 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:40.769394 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:40.769394 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 23:31:40.769394 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 23:31:40.769394 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 23:31:40.769394 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 23:31:40.769394 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:40.769394 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:40.769394 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:40.769394 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:40.769394 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:40.769394 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:40.770369 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:40.770369 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:40.770369 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:40.770369 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:40.770369 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:40.770369 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:40.770369 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:40.770369 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:40.770369 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:40.770369 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:40.770369 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 23:31:40.770369 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:40.770369 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 23:31:40.770369 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:40.770369 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 443/889] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Aug 17 23:31:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:40.771948 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183d0e9f-0747-4885-a390-4938e8c664c3 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:40.810687 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0c953-efe7-48c1-82c3-bb8fa3770e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:40.811225 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0c953-efe7-48c1-82c3-bb8fa3770e98 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:40.811504 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0c953-efe7-48c1-82c3-bb8fa3770e98 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:40.811778 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0c953-efe7-48c1-82c3-bb8fa3770e98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:40.873141 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:40.881700 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183d0e9f-0747-4885-a390-4938e8c664c3 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:40.892159 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be0c953-efe7-48c1-82c3-bb8fa3770e98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:40.894193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be0c953-efe7-48c1-82c3-bb8fa3770e98 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:40.919367 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f4880-684b-4ffd-b398-bcb02e01c96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:40.920051 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f4880-684b-4ffd-b398-bcb02e01c96f None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:40.920300 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f4880-684b-4ffd-b398-bcb02e01c96f None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:40.920574 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f4880-684b-4ffd-b398-bcb02e01c96f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:40.921162 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0f4880-684b-4ffd-b398-bcb02e01c96f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:40.922539 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0f4880-684b-4ffd-b398-bcb02e01c96f None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:40.931236 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-9b0f4880-684b-4ffd-b398-bcb02e01c96f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 17 23:31:40.931236 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:40.931236 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 23:31:40.931236 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 23:31:40.931236 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 23:31:40.931236 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 23:31:40.931236 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 23:31:40.931236 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:40.931236 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 17 23:31:40.931236 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:40.931236 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 23:31:40.931236 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:40.931236 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:40.931236 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:40.931236 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:40.931236 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:40.931236 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:40.932300 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:40.932300 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:40.932300 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:40.932300 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:40.932300 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:40.932300 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:40.932300 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 23:31:40.932300 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 23:31:40.932300 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 23:31:40.932300 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 23:31:40.932300 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:40.932300 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:40.932300 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:40.932300 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:40.932300 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:40.932300 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:40.933709 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:40.933709 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:40.933709 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:40.933709 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:40.933709 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:40.933709 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:40.933709 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:40.933709 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:40.933709 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:40.933709 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:40.933709 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 23:31:40.933709 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:40.933709 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 17 23:31:40.933709 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:40.933709 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 444/890] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 17 23:31:40 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:40.934516 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f4880-684b-4ffd-b398-bcb02e01c96f None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:40.993935 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:41.018626 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 447/891] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:31:40 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 23:31:41.019484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0c953-efe7-48c1-82c3-bb8fa3770e98 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:41.044796 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f4880-684b-4ffd-b398-bcb02e01c96f None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:41.087219 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a4f48-03bc-438c-a776-c3f36e2ec7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:41.088764 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a4f48-03bc-438c-a776-c3f36e2ec7ce None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:41.089147 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a4f48-03bc-438c-a776-c3f36e2ec7ce None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:41.089420 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a4f48-03bc-438c-a776-c3f36e2ec7ce None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:41.090051 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9a4f48-03bc-438c-a776-c3f36e2ec7ce None admin] RBAC: Adding query filter params (name=object-store) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:41.090350 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9a4f48-03bc-438c-a776-c3f36e2ec7ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:41.091727 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9a4f48-03bc-438c-a776-c3f36e2ec7ce None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:41.101729 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 445/892] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Aug 17 23:31:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:41.101729 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a4f48-03bc-438c-a776-c3f36e2ec7ce None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:41.115992 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:41.129923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0c953-efe7-48c1-82c3-bb8fa3770e98 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:41.167457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c52775-185e-4d0c-9d6a-0afc6871ee69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:41.168092 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c52775-185e-4d0c-9d6a-0afc6871ee69 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:41.168370 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c52775-185e-4d0c-9d6a-0afc6871ee69 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:41.168636 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c52775-185e-4d0c-9d6a-0afc6871ee69 None admin] PATH_INFO: `/v3/services` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:41.169261 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c52775-185e-4d0c-9d6a-0afc6871ee69 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:41.169562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c52775-185e-4d0c-9d6a-0afc6871ee69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:41.170927 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c52775-185e-4d0c-9d6a-0afc6871ee69 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:41.180083 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 448/893] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 17 23:31:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:41.180083 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c52775-185e-4d0c-9d6a-0afc6871ee69 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:41.213677 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a4f48-03bc-438c-a776-c3f36e2ec7ce None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:41.235077 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:41.251213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0a8d17-7178-472a-88f6-2eb4df6156c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:41.251907 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a8d17-7178-472a-88f6-2eb4df6156c3 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:41.252528 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a8d17-7178-472a-88f6-2eb4df6156c3 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:41.252528 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a8d17-7178-472a-88f6-2eb4df6156c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:41.264747 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0a8d17-7178-472a-88f6-2eb4df6156c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:41.266156 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0a8d17-7178-472a-88f6-2eb4df6156c3 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:41.281006 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-ea0a8d17-7178-472a-88f6-2eb4df6156c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:41.281006 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:41.281006 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:41.281006 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:41.281006 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:41.281006 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:41.281006 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:41.281006 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:41.281006 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:41.281006 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:41.281006 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:41.281006 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:41.281006 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:41.281006 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:41.281006 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:41.281006 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:41.282018 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:41.282018 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:41.282018 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:41.282018 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:41.282018 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:41.282018 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:41.282018 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:41.282018 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:41.282018 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:41.282018 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:41.282018 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:41.282018 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:41.282018 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:41.282018 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:41.282018 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:41.282018 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:41.282018 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:41.282932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:41.282932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:41.282932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:41.282932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 446/894] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:41.282932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0a8d17-7178-472a-88f6-2eb4df6156c3 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:41.292664 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c52775-185e-4d0c-9d6a-0afc6871ee69 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:41.328777 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8641acd-0126-4709-a812-c07d01dadf68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:41.329383 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8641acd-0126-4709-a812-c07d01dadf68 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:41.329648 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8641acd-0126-4709-a812-c07d01dadf68 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:41.329909 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8641acd-0126-4709-a812-c07d01dadf68 None admin] PATH_INFO: `/v3/services` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:41.330528 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8641acd-0126-4709-a812-c07d01dadf68 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:41.330824 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8641acd-0126-4709-a812-c07d01dadf68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:41.332272 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8641acd-0126-4709-a812-c07d01dadf68 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:41.344082 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 449/895] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Aug 17 23:31:41 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:41.344082 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8641acd-0126-4709-a812-c07d01dadf68 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:41.375011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:41.408161 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0a8d17-7178-472a-88f6-2eb4df6156c3 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:41.459042 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e132665-dfe8-4cd2-812f-35a6085b48ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:41.459668 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e132665-dfe8-4cd2-812f-35a6085b48ac None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:41.459988 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e132665-dfe8-4cd2-812f-35a6085b48ac None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:41.460232 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e132665-dfe8-4cd2-812f-35a6085b48ac None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:41.460874 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e132665-dfe8-4cd2-812f-35a6085b48ac None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:41.461172 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e132665-dfe8-4cd2-812f-35a6085b48ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:41.462825 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e132665-dfe8-4cd2-812f-35a6085b48ac None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:41.468924 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 447/896] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 17 23:31:41 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:41.477053 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e132665-dfe8-4cd2-812f-35a6085b48ac None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:41.485972 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8641acd-0126-4709-a812-c07d01dadf68 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:41.519375 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:41.526065 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc125d4-6b7c-4b07-bed9-4898a9609c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:41.526901 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc125d4-6b7c-4b07-bed9-4898a9609c70 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:41.527331 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc125d4-6b7c-4b07-bed9-4898a9609c70 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:41.528093 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc125d4-6b7c-4b07-bed9-4898a9609c70 None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:41.528748 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc125d4-6b7c-4b07-bed9-4898a9609c70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:41.529053 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc125d4-6b7c-4b07-bed9-4898a9609c70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:41.530426 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc125d4-6b7c-4b07-bed9-4898a9609c70 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:41.543231 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 450/897] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:41.548185 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc125d4-6b7c-4b07-bed9-4898a9609c70 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:41.604074 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e132665-dfe8-4cd2-812f-35a6085b48ac None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:41.640744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25cea7-0ef8-4c48-aabd-d6588aa35086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:41.641550 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25cea7-0ef8-4c48-aabd-d6588aa35086 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:41.641930 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25cea7-0ef8-4c48-aabd-d6588aa35086 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:41.642302 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25cea7-0ef8-4c48-aabd-d6588aa35086 None admin] PATH_INFO: `/v3/endpoints` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:41.643182 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb25cea7-0ef8-4c48-aabd-d6588aa35086 None admin] RBAC: Adding query filter params (interface=public, service_id=bd9b1b23392c48b58d2eaabd0c014d33, region_id=RegionOne) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:41.643593 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb25cea7-0ef8-4c48-aabd-d6588aa35086 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:41.645218 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb25cea7-0ef8-4c48-aabd-d6588aa35086 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:41.646894 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:41.660063 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 448/898] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Aug 17 23:31:41 2022] GET /identity/v3/endpoints?service_id=bd9b1b23392c48b58d2eaabd0c014d33&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:41.662600 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25cea7-0ef8-4c48-aabd-d6588aa35086 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:41.675058 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc125d4-6b7c-4b07-bed9-4898a9609c70 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:41.722460 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e111a-03b9-429c-bc1e-56ba667e8101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:41.723104 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e111a-03b9-429c-bc1e-56ba667e8101 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:41.723387 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e111a-03b9-429c-bc1e-56ba667e8101 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:41.724047 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e111a-03b9-429c-bc1e-56ba667e8101 None admin] PATH_INFO: `/v3/projects/service` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:41.734548 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e111a-03b9-429c-bc1e-56ba667e8101 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:41.736270 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e111a-03b9-429c-bc1e-56ba667e8101 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:41.748717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-746e111a-03b9-429c-bc1e-56ba667e8101 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:41.748717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:41.748717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:41.748717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:41.748717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:41.748717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:41.748717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:41.748717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:41.748717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:41.748717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:41.748717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:41.748717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:41.748717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 23:31:41.748717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 23:31:41.748717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 23:31:41.749638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 23:31:41.749638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:41.749638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:41.749638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:41.749638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:41.749638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:41.749638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:41.749638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:41.749638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:41.749638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:41.749638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:41.749638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:41.749638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:41.749638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:41.749638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:41.749638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:41.749638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:41.750521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 23:31:41.750521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 23:31:41.750521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 23:31:41.750521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 23:31:41.750521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 23:31:41.750521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 23:31:41.750521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:41.750521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:41.757321 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 451/899] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:31:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:31:41.758489 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e111a-03b9-429c-bc1e-56ba667e8101 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:41.781496 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:41.785932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25cea7-0ef8-4c48-aabd-d6588aa35086 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:41.839297 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd7dce-4a68-4fa2-9549-a1e89f62d3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:41.840288 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd7dce-4a68-4fa2-9549-a1e89f62d3b0 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:41.840569 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd7dce-4a68-4fa2-9549-a1e89f62d3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:41.840841 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd7dce-4a68-4fa2-9549-a1e89f62d3b0 None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:41.841453 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efd7dce-4a68-4fa2-9549-a1e89f62d3b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:41.842827 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efd7dce-4a68-4fa2-9549-a1e89f62d3b0 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:41.854895 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 449/900] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Aug 17 23:31:41 2022] GET /identity/v3/services => generated 1268 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:31:41.884311 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e111a-03b9-429c-bc1e-56ba667e8101 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:41.926074 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32305e9-2f17-4e6f-8878-a14b01ef7e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:41.927570 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32305e9-2f17-4e6f-8878-a14b01ef7e27 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:41.928282 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32305e9-2f17-4e6f-8878-a14b01ef7e27 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:41.928826 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32305e9-2f17-4e6f-8878-a14b01ef7e27 None admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:41.929771 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32305e9-2f17-4e6f-8878-a14b01ef7e27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:41.930341 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32305e9-2f17-4e6f-8878-a14b01ef7e27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:41.932089 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32305e9-2f17-4e6f-8878-a14b01ef7e27 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:41.935452 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:41.947789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 452/901] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 17 23:31:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:41.953531 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd7dce-4a68-4fa2-9549-a1e89f62d3b0 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:42.070065 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd7dce-4a68-4fa2-9549-a1e89f62d3b0 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:42.106738 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db0a158-9cfd-43bf-935a-99a54c371fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:42.107273 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0a158-9cfd-43bf-935a-99a54c371fda None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:42.107555 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0a158-9cfd-43bf-935a-99a54c371fda None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:42.107826 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0a158-9cfd-43bf-935a-99a54c371fda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:42.108587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db0a158-9cfd-43bf-935a-99a54c371fda None admin] RBAC: Adding query filter params (role.id=dc16b44096d040dfb737c287986be85e, scope.project.id=4de86915ef4340248ba8a8f51d56594e, user.id=d0a13dbffb4a49508b7fb661cf84e803) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:42.108896 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db0a158-9cfd-43bf-935a-99a54c371fda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:42.110333 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db0a158-9cfd-43bf-935a-99a54c371fda None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:42.116359 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 450/902] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:31:41 2022] GET /identity/v3/role_assignments?user.id=d0a13dbffb4a49508b7fb661cf84e803&scope.project.id=4de86915ef4340248ba8a8f51d56594e&role.id=dc16b44096d040dfb737c287986be85e => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:42.615500 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a496e-0953-4f49-a34d-93da655c22a0 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:42.616040 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a496e-0953-4f49-a34d-93da655c22a0 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:42.616122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a496e-0953-4f49-a34d-93da655c22a0 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:42.617687 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 453/903] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:42.623523 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daec7df-f2ab-4af6-bb23-8301ba07aa2f None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:42.623924 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daec7df-f2ab-4af6-bb23-8301ba07aa2f None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:42.624115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daec7df-f2ab-4af6-bb23-8301ba07aa2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:42.642188 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-8daec7df-f2ab-4af6-bb23-8301ba07aa2f None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:42.643375 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-8daec7df-f2ab-4af6-bb23-8301ba07aa2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:42.667853 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:42.720075 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 451/904] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:42 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:31:42.984631 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8a496e-0953-4f49-a34d-93da655c22a0 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:43.093462 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8a496e-0953-4f49-a34d-93da655c22a0 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:43.129582 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c550106-f2e2-482c-b58d-59ca0778e0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:43.130216 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c550106-f2e2-482c-b58d-59ca0778e0cc None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:43.130495 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c550106-f2e2-482c-b58d-59ca0778e0cc None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:43.130776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c550106-f2e2-482c-b58d-59ca0778e0cc None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:43.131360 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c550106-f2e2-482c-b58d-59ca0778e0cc None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:43.132751 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c550106-f2e2-482c-b58d-59ca0778e0cc None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:43.142163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-1c550106-f2e2-482c-b58d-59ca0778e0cc None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 17 23:31:43.142163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:43.142163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 23:31:43.142163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 23:31:43.142163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 23:31:43.142163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 23:31:43.142163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 23:31:43.142163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:43.142163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 17 23:31:43.142163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:43.142163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 23:31:43.142163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:43.142163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:43.142163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:43.142163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:43.142163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:43.143073 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:43.143073 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:43.143073 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:43.143073 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:43.143073 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:43.143073 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:43.143073 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:43.143073 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 23:31:43.143073 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 23:31:43.143073 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 23:31:43.143073 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 23:31:43.143073 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:43.143073 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:43.143073 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:43.143073 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:43.143073 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:43.143073 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:43.144023 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:43.144023 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:43.144023 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:43.144023 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:43.144023 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:43.144023 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:43.144023 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:43.144023 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:43.144023 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:43.144023 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:43.144023 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 23:31:43.144023 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:43.144023 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 17 23:31:43.144023 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:43.148181 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 454/905] 10.222.0.26 () {56 vars in 1132 bytes} [Wed Aug 17 23:31:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:43.149565 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daec7df-f2ab-4af6-bb23-8301ba07aa2f None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:43.255404 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daec7df-f2ab-4af6-bb23-8301ba07aa2f None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:43.305010 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00edced2-c463-451e-9de2-8538f6c8294d None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:43.305395 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00edced2-c463-451e-9de2-8538f6c8294d None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:43.305669 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00edced2-c463-451e-9de2-8538f6c8294d None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:43.305906 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c541ed-394d-4434-ae2a-72c9d0817e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:43.306548 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c541ed-394d-4434-ae2a-72c9d0817e16 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:43.306826 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c541ed-394d-4434-ae2a-72c9d0817e16 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:43.307097 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c541ed-394d-4434-ae2a-72c9d0817e16 None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:43.307261 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 455/906] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 23:31:43.307738 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c541ed-394d-4434-ae2a-72c9d0817e16 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:43.308069 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c541ed-394d-4434-ae2a-72c9d0817e16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:43.309439 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c541ed-394d-4434-ae2a-72c9d0817e16 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:43.313472 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadfd17-4f5a-4523-b0c1-e9a60fabcef2 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:43.313909 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadfd17-4f5a-4523-b0c1-e9a60fabcef2 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:43.314302 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadfd17-4f5a-4523-b0c1-e9a60fabcef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:43.315546 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 452/907] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Aug 17 23:31:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:31:43.321699 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c541ed-394d-4434-ae2a-72c9d0817e16 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:43.340898 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-5aadfd17-4f5a-4523-b0c1-e9a60fabcef2 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:43.342618 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-5aadfd17-4f5a-4523-b0c1-e9a60fabcef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:43.348064 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:43.413932 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:43.432771 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c541ed-394d-4434-ae2a-72c9d0817e16 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:43.433195 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 456/908] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:43 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 23:31:43.464330 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a560137-2b6f-44be-8cff-530344088c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:43.464993 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a560137-2b6f-44be-8cff-530344088c0e None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:43.465270 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a560137-2b6f-44be-8cff-530344088c0e None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:43.465540 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a560137-2b6f-44be-8cff-530344088c0e None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:43.466172 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a560137-2b6f-44be-8cff-530344088c0e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:43.466475 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a560137-2b6f-44be-8cff-530344088c0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:43.467957 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a560137-2b6f-44be-8cff-530344088c0e None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:43.473294 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 453/909] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Aug 17 23:31:43 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:31:43.479073 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aadfd17-4f5a-4523-b0c1-e9a60fabcef2 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:43.583742 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aadfd17-4f5a-4523-b0c1-e9a60fabcef2 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:43.619877 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040de77-32e4-4dd0-9d19-9f8418e1b655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:43.621179 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040de77-32e4-4dd0-9d19-9f8418e1b655 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:43.621604 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040de77-32e4-4dd0-9d19-9f8418e1b655 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:43.622122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040de77-32e4-4dd0-9d19-9f8418e1b655 None admin] PATH_INFO: `/v3/endpoints` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:43.622962 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4040de77-32e4-4dd0-9d19-9f8418e1b655 None admin] RBAC: Adding query filter params (interface=public, service_id=1eb30f8f696c4b6da46c706fd4bded01, region_id=RegionOne) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:43.623411 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4040de77-32e4-4dd0-9d19-9f8418e1b655 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:43.624965 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4040de77-32e4-4dd0-9d19-9f8418e1b655 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:43.630741 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 457/910] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Aug 17 23:31:43 2022] GET /identity/v3/endpoints?service_id=1eb30f8f696c4b6da46c706fd4bded01&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:43.637242 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a560137-2b6f-44be-8cff-530344088c0e None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:43.722532 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040de77-32e4-4dd0-9d19-9f8418e1b655 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:43.728054 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:43.744642 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a560137-2b6f-44be-8cff-530344088c0e None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:43.780669 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79287657-5c3c-43b2-86e9-96682354f2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:43.781342 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79287657-5c3c-43b2-86e9-96682354f2ba None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:43.781622 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79287657-5c3c-43b2-86e9-96682354f2ba None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:43.781899 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79287657-5c3c-43b2-86e9-96682354f2ba None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:43.782706 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79287657-5c3c-43b2-86e9-96682354f2ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:43.784408 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79287657-5c3c-43b2-86e9-96682354f2ba None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:43.789595 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:43.796103 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 454/911] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Aug 17 23:31:43 2022] GET /identity/v3/services => generated 1268 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:31:43.832408 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06157caf-2697-4aee-8ebc-d85b8f846346 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:43.832719 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06157caf-2697-4aee-8ebc-d85b8f846346 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:43.832957 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06157caf-2697-4aee-8ebc-d85b8f846346 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:43.834671 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 455/912] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 23:31:43.840645 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a39b0-1bae-40aa-8d56-111f73a21dff None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:43.840932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a39b0-1bae-40aa-8d56-111f73a21dff None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:43.841195 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a39b0-1bae-40aa-8d56-111f73a21dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:43.852042 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040de77-32e4-4dd0-9d19-9f8418e1b655 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:43.860787 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-436a39b0-1bae-40aa-8d56-111f73a21dff None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:43.862151 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-436a39b0-1bae-40aa-8d56-111f73a21dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:43.893029 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711de901-ee22-4eab-9c0d-8005f9c2f0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:43.894131 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711de901-ee22-4eab-9c0d-8005f9c2f0bf None admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:43.894698 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711de901-ee22-4eab-9c0d-8005f9c2f0bf None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:43.895376 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711de901-ee22-4eab-9c0d-8005f9c2f0bf None admin] PATH_INFO: `/v3/services` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:43.896445 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711de901-ee22-4eab-9c0d-8005f9c2f0bf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:43.898150 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711de901-ee22-4eab-9c0d-8005f9c2f0bf None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:43.924204 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 458/913] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Aug 17 23:31:43 2022] POST /identity/v3/services => generated 241 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:31:43.953176 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 456/914] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 23:31:44.042934 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9758e-82d8-4cdc-a6c8-57e2aca6c1a1 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:44.043221 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9758e-82d8-4cdc-a6c8-57e2aca6c1a1 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:44.043492 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9758e-82d8-4cdc-a6c8-57e2aca6c1a1 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:44.045159 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 459/915] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:44.050629 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b019e27-8435-4245-a05f-eefacfb814c3 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:44.050908 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b019e27-8435-4245-a05f-eefacfb814c3 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:44.051179 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b019e27-8435-4245-a05f-eefacfb814c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:44.069852 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-0b019e27-8435-4245-a05f-eefacfb814c3 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:44.071028 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-0b019e27-8435-4245-a05f-eefacfb814c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:44.145997 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 457/916] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:31:44.229531 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9758e-82d8-4cdc-a6c8-57e2aca6c1a1 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:44.338066 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9758e-82d8-4cdc-a6c8-57e2aca6c1a1 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:44.352496 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:44.374298 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5d7df-fff4-4c1c-b927-266d54175e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:44.375256 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5d7df-fff4-4c1c-b927-266d54175e82 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:44.375744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5d7df-fff4-4c1c-b927-266d54175e82 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:44.376288 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5d7df-fff4-4c1c-b927-266d54175e82 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:44.377115 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab5d7df-fff4-4c1c-b927-266d54175e82 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:44.378791 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab5d7df-fff4-4c1c-b927-266d54175e82 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:44.388986 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-eab5d7df-fff4-4c1c-b927-266d54175e82 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 17 23:31:44.388986 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:44.388986 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 23:31:44.388986 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 23:31:44.388986 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 23:31:44.388986 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 23:31:44.388986 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 23:31:44.388986 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:44.388986 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 17 23:31:44.388986 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:44.388986 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 23:31:44.388986 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:44.388986 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:44.388986 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:44.388986 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:44.388986 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:44.388986 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:44.390008 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:44.390008 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:44.390008 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:44.390008 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:44.390008 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:44.390008 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:44.390008 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 23:31:44.390008 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 23:31:44.390008 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 23:31:44.390008 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 23:31:44.390008 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:44.390008 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:44.390008 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:44.390008 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:44.390008 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:44.390008 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:44.390838 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:44.390838 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:44.390838 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:44.390838 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:44.390838 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:44.390838 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:44.390838 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:44.390838 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:44.390838 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:44.390838 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:44.390838 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 23:31:44.390838 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:44.390838 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 17 23:31:44.390838 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:44.393552 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 460/917] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 17 23:31:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:31:44.399685 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b019e27-8435-4245-a05f-eefacfb814c3 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:44.409615 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:44.414971 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5d7df-fff4-4c1c-b927-266d54175e82 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:44.524953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b019e27-8435-4245-a05f-eefacfb814c3 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:44.548922 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5d7df-fff4-4c1c-b927-266d54175e82 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:44.571724 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6c8c1-2608-421a-b328-bf4bf8acc07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:44.572731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6c8c1-2608-421a-b328-bf4bf8acc07e None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:44.573426 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6c8c1-2608-421a-b328-bf4bf8acc07e None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:44.573896 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6c8c1-2608-421a-b328-bf4bf8acc07e None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:44.575172 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c6c8c1-2608-421a-b328-bf4bf8acc07e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:44.575716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c6c8c1-2608-421a-b328-bf4bf8acc07e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:44.577881 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c6c8c1-2608-421a-b328-bf4bf8acc07e None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:44.588298 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ede35b-6254-455e-a772-52fd56d91d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:44.588298 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ede35b-6254-455e-a772-52fd56d91d42 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:44.588298 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ede35b-6254-455e-a772-52fd56d91d42 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:44.588298 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ede35b-6254-455e-a772-52fd56d91d42 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:44.589224 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 458/918] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 17 23:31:44 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:44.590918 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6c8c1-2608-421a-b328-bf4bf8acc07e None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:44.604374 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ede35b-6254-455e-a772-52fd56d91d42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:31:44.611330 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-17ede35b-6254-455e-a772-52fd56d91d42 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 17 23:31:44.611330 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:44.611330 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:44.611330 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:44.611330 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:44.611330 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:44.611330 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:44.611330 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:44.611330 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:44.611330 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:44.611330 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:44.611330 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:44.611330 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:31:44.611330 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:31:44.611330 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:31:44.611330 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:31:44.612441 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:31:44.612441 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:31:44.612441 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:31:44.612441 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:31:44.612441 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:44.612441 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:44.612441 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:44.612441 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:44.612441 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:44.612441 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:44.612441 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:44.612441 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:44.612441 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:44.612441 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:44.612441 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:44.612441 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:44.613469 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:44.613469 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:44.613469 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:44.613469 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:44.613469 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:31:44.613469 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:31:44.613469 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:31:44.613469 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:31:44.613469 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:31:44.613469 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:31:44.613469 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 17 23:31:44.613469 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:44.616088 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 461/919] 10.222.0.26 () {56 vars in 1120 bytes} [Wed Aug 17 23:31:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:31:44.622209 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ede35b-6254-455e-a772-52fd56d91d42 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:44.674643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:44.678768 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:44.709897 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6c8c1-2608-421a-b328-bf4bf8acc07e None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:44.746163 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d19698-4548-4bf5-9ffb-3ccac06cdbd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:44.746782 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d19698-4548-4bf5-9ffb-3ccac06cdbd7 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:44.747063 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d19698-4548-4bf5-9ffb-3ccac06cdbd7 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:44.747338 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d19698-4548-4bf5-9ffb-3ccac06cdbd7 None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:44.748067 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d19698-4548-4bf5-9ffb-3ccac06cdbd7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:44.748373 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d19698-4548-4bf5-9ffb-3ccac06cdbd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:44.749756 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d19698-4548-4bf5-9ffb-3ccac06cdbd7 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:44.759314 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 459/920] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Aug 17 23:31:44 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:44.760270 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d19698-4548-4bf5-9ffb-3ccac06cdbd7 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:44.766117 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ede35b-6254-455e-a772-52fd56d91d42 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:44.807754 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93db37e-5698-40f5-9e8f-a8402a99fcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:44.808427 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93db37e-5698-40f5-9e8f-a8402a99fcbe None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:44.808708 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93db37e-5698-40f5-9e8f-a8402a99fcbe None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:44.808978 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93db37e-5698-40f5-9e8f-a8402a99fcbe None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:44.809647 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93db37e-5698-40f5-9e8f-a8402a99fcbe None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:44.809947 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93db37e-5698-40f5-9e8f-a8402a99fcbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:44.811306 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93db37e-5698-40f5-9e8f-a8402a99fcbe None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:44.826699 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 462/921] 10.222.0.26 () {56 vars in 1115 bytes} [Wed Aug 17 23:31:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:31:44.829440 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93db37e-5698-40f5-9e8f-a8402a99fcbe None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:44.886808 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d19698-4548-4bf5-9ffb-3ccac06cdbd7 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:44.928623 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7360af-5df7-4c13-9668-9eb2d926cf8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:44.929683 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7360af-5df7-4c13-9668-9eb2d926cf8d None admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:44.930174 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7360af-5df7-4c13-9668-9eb2d926cf8d None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:44.930672 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7360af-5df7-4c13-9668-9eb2d926cf8d None admin] PATH_INFO: `/v3/endpoints` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:44.931585 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7360af-5df7-4c13-9668-9eb2d926cf8d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:44.933340 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7360af-5df7-4c13-9668-9eb2d926cf8d None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:44.938622 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:44.951773 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:44.955527 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93db37e-5698-40f5-9e8f-a8402a99fcbe None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:44.961303 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 460/922] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:31:44 2022] POST /identity/v3/endpoints => generated 346 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:31:45.012685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88095356-4d45-4d41-8c5e-bf57b36812d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:45.013330 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88095356-4d45-4d41-8c5e-bf57b36812d1 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:45.013605 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88095356-4d45-4d41-8c5e-bf57b36812d1 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:45.013873 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88095356-4d45-4d41-8c5e-bf57b36812d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:45.027174 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88095356-4d45-4d41-8c5e-bf57b36812d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:45.029028 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88095356-4d45-4d41-8c5e-bf57b36812d1 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:45.046461 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-88095356-4d45-4d41-8c5e-bf57b36812d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:45.046461 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:45.046461 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:45.046461 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:45.046461 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:45.046461 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:45.046461 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:45.046461 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:45.046461 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:45.046461 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:45.046461 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:45.046461 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:45.046461 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:45.046461 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:45.046461 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:45.046461 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:45.047575 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:45.047575 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:45.047575 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:45.047575 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:45.047575 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:45.047575 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:45.047575 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:45.047575 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:45.047575 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:45.047575 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:45.047575 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:45.047575 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:45.047575 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:45.047575 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:45.047575 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:45.047575 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:45.047575 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:45.048502 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:45.048502 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:45.048502 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:45.048502 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 463/923] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:31:45.055034 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7360af-5df7-4c13-9668-9eb2d926cf8d None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:45.170203 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7360af-5df7-4c13-9668-9eb2d926cf8d None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:45.204783 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f4105e-9cc5-45f0-8035-06be9aa19b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:45.205311 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4105e-9cc5-45f0-8035-06be9aa19b17 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:45.205589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4105e-9cc5-45f0-8035-06be9aa19b17 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:45.205859 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4105e-9cc5-45f0-8035-06be9aa19b17 None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:45.206504 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f4105e-9cc5-45f0-8035-06be9aa19b17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:45.206808 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f4105e-9cc5-45f0-8035-06be9aa19b17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:45.208197 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f4105e-9cc5-45f0-8035-06be9aa19b17 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:45.220307 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 461/924] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:45.225930 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88095356-4d45-4d41-8c5e-bf57b36812d1 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:45.337279 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88095356-4d45-4d41-8c5e-bf57b36812d1 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:45.372899 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b4afd-d276-4104-9cda-676a7abb8480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:45.373947 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b4afd-d276-4104-9cda-676a7abb8480 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:45.374441 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b4afd-d276-4104-9cda-676a7abb8480 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:45.374924 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b4afd-d276-4104-9cda-676a7abb8480 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:45.383003 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8b4afd-d276-4104-9cda-676a7abb8480 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:45.384846 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8b4afd-d276-4104-9cda-676a7abb8480 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:45.396587 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-1d8b4afd-d276-4104-9cda-676a7abb8480 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 17 23:31:45.396587 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:45.396587 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:45.396587 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:45.396587 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:45.396587 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:45.396587 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:45.396587 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:45.396587 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:45.396587 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:45.396587 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:45.396587 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:45.396587 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 23:31:45.396587 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 23:31:45.396587 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 23:31:45.396587 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 23:31:45.397608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:45.397608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:45.397608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 23:31:45.397608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:45.397608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 23:31:45.397608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:45.397608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:45.397608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:45.397608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:45.397608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:45.397608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:45.397608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:45.397608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:45.397608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:45.397608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:45.397608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:45.397608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:45.397608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:45.398632 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:45.398632 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:45.398632 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 23:31:45.398632 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 23:31:45.398632 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 23:31:45.398632 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 23:31:45.398632 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 23:31:45.398632 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 23:31:45.398632 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 17 23:31:45.398632 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:45.401354 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 464/925] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Aug 17 23:31:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:45.407948 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f4105e-9cc5-45f0-8035-06be9aa19b17 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:45.476482 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:45.531523 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f4105e-9cc5-45f0-8035-06be9aa19b17 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:45.567168 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d3749-9fdf-4630-930a-c8bb08e4fb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:45.568173 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d3749-9fdf-4630-930a-c8bb08e4fb14 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:45.568665 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d3749-9fdf-4630-930a-c8bb08e4fb14 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:45.569145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d3749-9fdf-4630-930a-c8bb08e4fb14 None admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:45.570175 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d3749-9fdf-4630-930a-c8bb08e4fb14 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:45.570687 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d3749-9fdf-4630-930a-c8bb08e4fb14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:45.572413 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d3749-9fdf-4630-930a-c8bb08e4fb14 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:45.591149 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 462/926] 10.222.0.26 () {56 vars in 1149 bytes} [Wed Aug 17 23:31:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:45.597195 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b4afd-d276-4104-9cda-676a7abb8480 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:45.610474 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:45.703461 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b4afd-d276-4104-9cda-676a7abb8480 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:45.739718 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a266afbe-b5bb-495f-a17f-d4f5861dc099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:45.740290 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266afbe-b5bb-495f-a17f-d4f5861dc099 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:45.740564 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266afbe-b5bb-495f-a17f-d4f5861dc099 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:45.740831 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266afbe-b5bb-495f-a17f-d4f5861dc099 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:45.751342 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a266afbe-b5bb-495f-a17f-d4f5861dc099 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:45.752789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a266afbe-b5bb-495f-a17f-d4f5861dc099 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:45.766677 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-a266afbe-b5bb-495f-a17f-d4f5861dc099 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:45.766677 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:45.766677 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:45.766677 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:45.766677 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:45.766677 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:45.766677 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:45.766677 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:45.766677 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:45.766677 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:45.766677 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:45.766677 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:45.766677 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:45.766677 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:45.766677 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:45.766677 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:45.767630 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:45.767630 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:45.767630 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:45.767630 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:45.767630 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:45.767630 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:45.767630 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:45.767630 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:45.767630 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:45.767630 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:45.767630 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:45.767630 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:45.767630 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:45.767630 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:45.767630 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:45.767630 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:45.767630 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:45.768560 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:45.768560 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:45.768560 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:45.772068 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 465/927] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:45.773770 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d3749-9fdf-4630-930a-c8bb08e4fb14 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:45.776339 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5fe6e-9eb9-4d24-9ce9-2662700dac22 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:45.776620 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5fe6e-9eb9-4d24-9ce9-2662700dac22 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:45.776888 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5fe6e-9eb9-4d24-9ce9-2662700dac22 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:45.778768 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 466/928] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:45.783476 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78c382-64c4-486c-8b4a-b63bfa8a0767 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:45.783754 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78c382-64c4-486c-8b4a-b63bfa8a0767 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:45.784066 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78c382-64c4-486c-8b4a-b63bfa8a0767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:45.806431 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-9f78c382-64c4-486c-8b4a-b63bfa8a0767 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:45.808510 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-9f78c382-64c4-486c-8b4a-b63bfa8a0767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:45.876857 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:45.893411 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 467/929] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 23:31:45.905241 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d3749-9fdf-4630-930a-c8bb08e4fb14 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:45.915214 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:45.942050 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c1076-ae87-460e-8b28-70919afc456e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:45.943025 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c1076-ae87-460e-8b28-70919afc456e None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:45.943515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c1076-ae87-460e-8b28-70919afc456e None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:45.944092 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c1076-ae87-460e-8b28-70919afc456e None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:45.944952 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c1076-ae87-460e-8b28-70919afc456e None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:45.945515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c1076-ae87-460e-8b28-70919afc456e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:45.946963 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc83dc3-9754-4372-8e8c-9b29e74cabf0 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:45.947252 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc83dc3-9754-4372-8e8c-9b29e74cabf0 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:45.947524 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc83dc3-9754-4372-8e8c-9b29e74cabf0 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:45.947889 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c1076-ae87-460e-8b28-70919afc456e None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:45.952613 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 468/930] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 23:31:45.954124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603debd-2070-4ca0-884a-8d91f68642b2 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:45.954474 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603debd-2070-4ca0-884a-8d91f68642b2 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:45.954821 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603debd-2070-4ca0-884a-8d91f68642b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:45.961559 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 463/931] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:45.969165 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c1076-ae87-460e-8b28-70919afc456e None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:45.973886 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-a603debd-2070-4ca0-884a-8d91f68642b2 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:45.975171 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-a603debd-2070-4ca0-884a-8d91f68642b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:46.060531 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 469/932] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 23:31:46.082405 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c1076-ae87-460e-8b28-70919afc456e None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:46.117132 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdcf893-59b7-4317-9e0b-84e261aab9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:46.118068 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdcf893-59b7-4317-9e0b-84e261aab9ce None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:46.118533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdcf893-59b7-4317-9e0b-84e261aab9ce None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:46.119013 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdcf893-59b7-4317-9e0b-84e261aab9ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:46.125853 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdcf893-59b7-4317-9e0b-84e261aab9ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:46.127477 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdcf893-59b7-4317-9e0b-84e261aab9ce None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:46.139214 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-7bdcf893-59b7-4317-9e0b-84e261aab9ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:46.139214 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:46.139214 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:46.139214 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:46.139214 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:46.139214 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:46.139214 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:46.139214 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:46.139214 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:46.139214 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:46.139214 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:46.139214 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:46.139214 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 23:31:46.139214 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 23:31:46.139214 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 23:31:46.140163 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 23:31:46.140163 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:46.140163 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:46.140163 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:46.140163 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:46.140163 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:46.140163 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:46.140163 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:46.140163 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:46.140163 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:46.140163 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:46.140163 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:46.140163 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:46.140163 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:46.140163 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:46.140163 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:46.140163 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:46.141066 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 23:31:46.141066 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 23:31:46.141066 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 23:31:46.141066 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 23:31:46.141066 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 23:31:46.141066 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 23:31:46.141066 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:46.141066 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:46.143078 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 464/933] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:31:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:31:46.148460 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603debd-2070-4ca0-884a-8d91f68642b2 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:46.179366 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdcf893-59b7-4317-9e0b-84e261aab9ce None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:46.216214 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:46.279341 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603debd-2070-4ca0-884a-8d91f68642b2 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:46.296599 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdcf893-59b7-4317-9e0b-84e261aab9ce None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:46.305705 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:46.326849 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8659c5e0-c85f-46f4-a9ca-c4b91ad8a41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:46.327384 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659c5e0-c85f-46f4-a9ca-c4b91ad8a41b None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:46.327698 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659c5e0-c85f-46f4-a9ca-c4b91ad8a41b None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:46.328466 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659c5e0-c85f-46f4-a9ca-c4b91ad8a41b None admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:46.329283 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8659c5e0-c85f-46f4-a9ca-c4b91ad8a41b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:46.329724 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8659c5e0-c85f-46f4-a9ca-c4b91ad8a41b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:46.331407 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8659c5e0-c85f-46f4-a9ca-c4b91ad8a41b None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:46.350095 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 470/934] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 17 23:31:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:46.350095 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8659c5e0-c85f-46f4-a9ca-c4b91ad8a41b None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:46.351096 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed5746-cbe4-41f3-ba42-c84cadfc799a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:46.351778 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed5746-cbe4-41f3-ba42-c84cadfc799a None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:46.352092 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed5746-cbe4-41f3-ba42-c84cadfc799a None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:46.352456 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed5746-cbe4-41f3-ba42-c84cadfc799a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:46.353604 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed5746-cbe4-41f3-ba42-c84cadfc799a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:46.355020 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed5746-cbe4-41f3-ba42-c84cadfc799a None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:46.364253 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-e7ed5746-cbe4-41f3-ba42-c84cadfc799a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 17 23:31:46.364253 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:46.364253 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 23:31:46.364253 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 23:31:46.364253 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 23:31:46.364253 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 23:31:46.364253 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 23:31:46.364253 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:46.364253 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 17 23:31:46.364253 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:46.364253 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 23:31:46.364253 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:46.364253 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:46.364253 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:46.364253 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:46.364253 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:46.365266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:46.365266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:46.365266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:46.365266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:46.365266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:46.365266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:46.365266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:46.365266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 23:31:46.365266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 23:31:46.365266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 23:31:46.365266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 23:31:46.365266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:46.365266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:46.365266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:46.365266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:46.365266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:46.365266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:46.366190 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:46.366190 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:46.366190 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:46.366190 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:46.366190 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:46.366190 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:46.366190 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:46.366190 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:46.366190 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:46.366190 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:46.366190 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 23:31:46.366190 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:46.366190 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 17 23:31:46.366190 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:46.366190 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 465/935] 10.222.0.26 () {56 vars in 1132 bytes} [Wed Aug 17 23:31:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:31:46.367012 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed5746-cbe4-41f3-ba42-c84cadfc799a None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:46.470407 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:46.471828 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8659c5e0-c85f-46f4-a9ca-c4b91ad8a41b None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:46.481330 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed5746-cbe4-41f3-ba42-c84cadfc799a None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:46.507481 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3b50d-2851-448b-840b-9efe9488ccdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:46.508145 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3b50d-2851-448b-840b-9efe9488ccdc None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:46.508429 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3b50d-2851-448b-840b-9efe9488ccdc None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:46.508702 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3b50d-2851-448b-840b-9efe9488ccdc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:46.509389 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3b50d-2851-448b-840b-9efe9488ccdc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:46.510671 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3b50d-2851-448b-840b-9efe9488ccdc None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:46.516597 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f38335-954e-46bc-aec7-fd86e4e30758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:46.517250 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f38335-954e-46bc-aec7-fd86e4e30758 None admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:46.517526 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f38335-954e-46bc-aec7-fd86e4e30758 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:46.517796 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f38335-954e-46bc-aec7-fd86e4e30758 None admin] PATH_INFO: `/v3/projects/4de86915ef4340248ba8a8f51d56594e/users/d0a13dbffb4a49508b7fb661cf84e803/roles/dc16b44096d040dfb737c287986be85e` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:46.519908 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f38335-954e-46bc-aec7-fd86e4e30758 None admin] RBAC: Authorizing `identity:create_grant(project_id=4de86915ef4340248ba8a8f51d56594e, user_id=d0a13dbffb4a49508b7fb661cf84e803, role_id=dc16b44096d040dfb737c287986be85e)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:46.520538 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-f0d3b50d-2851-448b-840b-9efe9488ccdc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 23:31:46.520538 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:46.520538 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 23:31:46.520538 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 23:31:46.520538 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 23:31:46.520538 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 23:31:46.520538 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 23:31:46.520538 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:46.520538 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 23:31:46.520538 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:46.520538 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 23:31:46.520538 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:46.520538 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:46.520538 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:46.520538 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:46.520538 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:46.520538 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:46.521543 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:46.521543 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:46.521543 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:46.521543 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:46.521543 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:46.521543 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:46.521543 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 23:31:46.521543 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 23:31:46.521543 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 23:31:46.521543 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 23:31:46.521543 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:46.521543 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:46.521543 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:46.521543 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:46.521543 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:46.521543 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:46.522388 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:46.522388 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:46.522388 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:46.522388 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:46.522388 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:46.522388 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:46.522388 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:46.522388 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:46.522388 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:46.522388 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:46.522388 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 23:31:46.522388 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:46.522388 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 23:31:46.522388 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:46.523118 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f38335-954e-46bc-aec7-fd86e4e30758 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:46.526122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 471/936] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Aug 17 23:31:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:46.526769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3b50d-2851-448b-840b-9efe9488ccdc None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:46.538196 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 466/937] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 17 23:31:46 2022] PUT /identity/v3/projects/4de86915ef4340248ba8a8f51d56594e/users/d0a13dbffb4a49508b7fb661cf84e803/roles/dc16b44096d040dfb737c287986be85e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:31:46.538791 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f38335-954e-46bc-aec7-fd86e4e30758 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:46.590744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:46.657753 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3b50d-2851-448b-840b-9efe9488ccdc None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:46.691667 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f38335-954e-46bc-aec7-fd86e4e30758 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:46.697966 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026b8680-d997-4874-8c81-e24a3aef506d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:46.698584 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b8680-d997-4874-8c81-e24a3aef506d None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:46.698861 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b8680-d997-4874-8c81-e24a3aef506d None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:46.699215 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b8680-d997-4874-8c81-e24a3aef506d None admin] PATH_INFO: `/v3/services` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:46.699852 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026b8680-d997-4874-8c81-e24a3aef506d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:46.700243 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026b8680-d997-4874-8c81-e24a3aef506d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:46.701681 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026b8680-d997-4874-8c81-e24a3aef506d None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:46.707327 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 472/938] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Aug 17 23:31:46 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:46.714258 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026b8680-d997-4874-8c81-e24a3aef506d None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:46.736285 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de9060e-2350-42c4-bdb3-e493687e9973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:46.737149 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de9060e-2350-42c4-bdb3-e493687e9973 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:46.737427 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de9060e-2350-42c4-bdb3-e493687e9973 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:46.738122 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de9060e-2350-42c4-bdb3-e493687e9973 None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:46.738768 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de9060e-2350-42c4-bdb3-e493687e9973 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:46.739128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de9060e-2350-42c4-bdb3-e493687e9973 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:46.740550 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de9060e-2350-42c4-bdb3-e493687e9973 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:46.748292 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 467/939] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Aug 17 23:31:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:46.753569 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de9060e-2350-42c4-bdb3-e493687e9973 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:46.773054 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:46.828738 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026b8680-d997-4874-8c81-e24a3aef506d None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:46.845748 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:46.864437 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a09d5-bb6d-46ee-9ed5-f6fb33d39df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:46.865427 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a09d5-bb6d-46ee-9ed5-f6fb33d39df6 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:46.865942 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a09d5-bb6d-46ee-9ed5-f6fb33d39df6 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:46.866465 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a09d5-bb6d-46ee-9ed5-f6fb33d39df6 None admin] PATH_INFO: `/v3/services` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:46.867304 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a09d5-bb6d-46ee-9ed5-f6fb33d39df6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:46.867814 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a09d5-bb6d-46ee-9ed5-f6fb33d39df6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:46.869444 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a09d5-bb6d-46ee-9ed5-f6fb33d39df6 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:46.874154 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de9060e-2350-42c4-bdb3-e493687e9973 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:46.875732 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 473/940] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Aug 17 23:31:46 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:46.882395 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a09d5-bb6d-46ee-9ed5-f6fb33d39df6 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:46.920716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a6c7e4-a551-442a-b2b7-8df245077a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:46.921662 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6c7e4-a551-442a-b2b7-8df245077a85 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:46.921943 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6c7e4-a551-442a-b2b7-8df245077a85 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:46.922255 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6c7e4-a551-442a-b2b7-8df245077a85 None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:46.922888 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a6c7e4-a551-442a-b2b7-8df245077a85 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:46.923229 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a6c7e4-a551-442a-b2b7-8df245077a85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:46.925171 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a6c7e4-a551-442a-b2b7-8df245077a85 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:46.936037 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 468/941] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Aug 17 23:31:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:31:46.937620 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a6c7e4-a551-442a-b2b7-8df245077a85 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:47.013614 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a09d5-bb6d-46ee-9ed5-f6fb33d39df6 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:47.042187 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:47.050049 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edccdc5-075e-46dd-b3e6-1530c2594014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:47.050763 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edccdc5-075e-46dd-b3e6-1530c2594014 None admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:47.051105 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edccdc5-075e-46dd-b3e6-1530c2594014 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:47.051850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edccdc5-075e-46dd-b3e6-1530c2594014 None admin] PATH_INFO: `/v3/endpoints` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:47.052617 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edccdc5-075e-46dd-b3e6-1530c2594014 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:47.054150 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edccdc5-075e-46dd-b3e6-1530c2594014 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:47.064322 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a6c7e4-a551-442a-b2b7-8df245077a85 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:47.082930 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 474/942] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:31:46 2022] POST /identity/v3/endpoints => generated 347 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:31:47.082930 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c915c67-d692-42b9-93db-f54d3af18f0c None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:47.082930 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c915c67-d692-42b9-93db-f54d3af18f0c None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:47.082930 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c915c67-d692-42b9-93db-f54d3af18f0c None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:47.085565 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 475/943] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:47.088476 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6385deb-fa0e-45b9-b0a7-7853a082eada None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:47.088894 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6385deb-fa0e-45b9-b0a7-7853a082eada None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:47.089168 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6385deb-fa0e-45b9-b0a7-7853a082eada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:47.113620 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-a6385deb-fa0e-45b9-b0a7-7853a082eada None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:47.114871 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-a6385deb-fa0e-45b9-b0a7-7853a082eada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:47.115065 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600d1bd-da4e-4613-9d83-690ff6cbda0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:47.115667 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600d1bd-da4e-4613-9d83-690ff6cbda0e None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:47.115949 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600d1bd-da4e-4613-9d83-690ff6cbda0e None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:47.116249 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600d1bd-da4e-4613-9d83-690ff6cbda0e None admin] PATH_INFO: `/v3/endpoints` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:47.117177 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5600d1bd-da4e-4613-9d83-690ff6cbda0e None admin] RBAC: Adding query filter params (interface=public, service_id=6270541d709148648c5e544ed6d276ac, region_id=RegionOne) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:47.117357 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5600d1bd-da4e-4613-9d83-690ff6cbda0e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:47.119573 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5600d1bd-da4e-4613-9d83-690ff6cbda0e None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:47.130660 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 469/944] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Aug 17 23:31:46 2022] GET /identity/v3/endpoints?service_id=6270541d709148648c5e544ed6d276ac&interface=public®ion_id=RegionOne => generated 194 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:31:47.131614 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600d1bd-da4e-4613-9d83-690ff6cbda0e None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:47.144810 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:47.219991 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 476/945] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 23:31:47.257415 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600d1bd-da4e-4613-9d83-690ff6cbda0e None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:47.293563 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd0fdf0-b94f-4f7a-8605-52c6da34d46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:47.294222 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0fdf0-b94f-4f7a-8605-52c6da34d46f None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:47.294506 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0fdf0-b94f-4f7a-8605-52c6da34d46f None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:47.294781 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0fdf0-b94f-4f7a-8605-52c6da34d46f None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:47.295391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd0fdf0-b94f-4f7a-8605-52c6da34d46f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:47.296865 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd0fdf0-b94f-4f7a-8605-52c6da34d46f None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:47.303452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 470/946] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Aug 17 23:31:47 2022] GET /identity/v3/services => generated 1497 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:31:47.486801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6385deb-fa0e-45b9-b0a7-7853a082eada None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:47.589468 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6385deb-fa0e-45b9-b0a7-7853a082eada None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:47.626798 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d2d7a-7ede-4e9b-b50a-024731e81673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:47.627415 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d2d7a-7ede-4e9b-b50a-024731e81673 None admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:47.627698 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d2d7a-7ede-4e9b-b50a-024731e81673 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:47.627991 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d2d7a-7ede-4e9b-b50a-024731e81673 None admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:47.628660 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d2d7a-7ede-4e9b-b50a-024731e81673 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:47.630098 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d2d7a-7ede-4e9b-b50a-024731e81673 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:47.633377 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: WARNING keystone.server.flask.common [None req-923d2d7a-7ede-4e9b-b50a-024731e81673 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 17 23:31:47.651246 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.sql.core [None req-923d2d7a-7ede-4e9b-b50a-024731e81673 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 17 23:31:47.651246 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 17 23:31:47.651246 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [parameters: {'id': 'a00ea0f80a4a487fac89ebc4b61f430c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 17 23:31:47.651246 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=85834) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 17 23:31:47.669307 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-923d2d7a-7ede-4e9b-b50a-024731e81673 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a00ea0f80a4a487fac89ebc4b61f430c.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a00ea0f80a4a487fac89ebc4b61f430c. Aug 17 23:31:47.669307 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:47.669307 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 17 23:31:47.669307 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 17 23:31:47.669307 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 17 23:31:47.669307 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 17 23:31:47.669307 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 17 23:31:47.669307 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application result = self._query(query) Aug 17 23:31:47.669307 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 17 23:31:47.669307 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application conn.query(q) Aug 17 23:31:47.669307 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 17 23:31:47.669307 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 17 23:31:47.669307 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 17 23:31:47.670193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application result.read() Aug 17 23:31:47.670193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 17 23:31:47.670193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 17 23:31:47.670193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 17 23:31:47.670193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 17 23:31:47.670193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 17 23:31:47.670193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 17 23:31:47.670193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 17 23:31:47.670193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 17 23:31:47.670193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 17 23:31:47.670193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:47.670193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 17 23:31:47.670193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:47.670193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:47.670193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 17 23:31:47.670193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 17 23:31:47.670193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 17 23:31:47.670193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 17 23:31:47.671163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 17 23:31:47.671163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application next(self.gen) Aug 17 23:31:47.671163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 17 23:31:47.671163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application yield resource Aug 17 23:31:47.671163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 17 23:31:47.671163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application next(self.gen) Aug 17 23:31:47.671163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 17 23:31:47.671163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application self.session.rollback() Aug 17 23:31:47.671163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 17 23:31:47.671163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application self.force_reraise() Aug 17 23:31:47.671163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 17 23:31:47.671163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise self.value Aug 17 23:31:47.671163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 17 23:31:47.671163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 17 23:31:47.671163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 17 23:31:47.671163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application session.commit() Aug 17 23:31:47.671163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 17 23:31:47.672106 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 17 23:31:47.672106 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 17 23:31:47.672106 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application self._prepare_impl() Aug 17 23:31:47.672106 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 17 23:31:47.672106 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application self.session.flush() Aug 17 23:31:47.672106 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 17 23:31:47.672106 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application self._flush(objects) Aug 17 23:31:47.672106 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 17 23:31:47.672106 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 17 23:31:47.672106 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 17 23:31:47.672106 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application compat.raise_( Aug 17 23:31:47.672106 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 17 23:31:47.672106 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception Aug 17 23:31:47.672106 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 17 23:31:47.672106 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application flush_context.execute() Aug 17 23:31:47.672106 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 17 23:31:47.672106 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rec.execute(self) Aug 17 23:31:47.673032 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 17 23:31:47.673032 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 17 23:31:47.673032 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 17 23:31:47.673032 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 17 23:31:47.673032 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 17 23:31:47.673032 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 17 23:31:47.673032 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 17 23:31:47.673032 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 17 23:31:47.673032 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 17 23:31:47.673032 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 17 23:31:47.673032 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 17 23:31:47.673032 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 17 23:31:47.673032 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 17 23:31:47.673032 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 17 23:31:47.673032 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 17 23:31:47.673850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 17 23:31:47.673850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 17 23:31:47.673850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception Aug 17 23:31:47.673850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 17 23:31:47.673850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 17 23:31:47.673850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 17 23:31:47.673850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 17 23:31:47.673850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 17 23:31:47.673850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application result = self._query(query) Aug 17 23:31:47.673850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 17 23:31:47.673850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application conn.query(q) Aug 17 23:31:47.673850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 17 23:31:47.673850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 17 23:31:47.673850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 17 23:31:47.673850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application result.read() Aug 17 23:31:47.673850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 17 23:31:47.673850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 17 23:31:47.674744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 17 23:31:47.674744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 17 23:31:47.674744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 17 23:31:47.674744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 17 23:31:47.674744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 17 23:31:47.674744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 17 23:31:47.674744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 17 23:31:47.674744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 17 23:31:47.674744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [parameters: {'id': 'a00ea0f80a4a487fac89ebc4b61f430c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 17 23:31:47.674744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 17 23:31:47.674744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:47.674744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 23:31:47.674744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:47.674744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:47.674744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 17 23:31:47.675562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 17 23:31:47.675562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 17 23:31:47.675562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 17 23:31:47.675562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 17 23:31:47.675562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:47.675562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 23:31:47.675562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:47.675562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:47.675562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:47.675562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:47.675562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:47.675562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:47.675562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:47.675562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:47.675562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:47.675562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:47.675562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:47.682130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:47.682130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 17 23:31:47.682130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 17 23:31:47.682130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:47.682130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:47.682130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 17 23:31:47.682130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 17 23:31:47.682130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a00ea0f80a4a487fac89ebc4b61f430c. Aug 17 23:31:47.682130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:47.682130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 477/947] 10.222.0.26 () {60 vars in 1140 bytes} [Wed Aug 17 23:31:47 2022] POST /identity/v3/projects => generated 262 bytes in 185 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:47.682837 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd0fdf0-b94f-4f7a-8605-52c6da34d46f None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:47.688575 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:47.793765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd0fdf0-b94f-4f7a-8605-52c6da34d46f None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:47.829127 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6580fe2-a08d-4e34-9270-f16e97b5d3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:47.830083 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6580fe2-a08d-4e34-9270-f16e97b5d3ec None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:47.830570 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6580fe2-a08d-4e34-9270-f16e97b5d3ec None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:47.831046 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6580fe2-a08d-4e34-9270-f16e97b5d3ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:47.838115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6580fe2-a08d-4e34-9270-f16e97b5d3ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:47.839754 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6580fe2-a08d-4e34-9270-f16e97b5d3ec None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:47.851045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-a6580fe2-a08d-4e34-9270-f16e97b5d3ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:47.851045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:47.851045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:47.851045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:47.851045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:47.851045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:47.851045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:47.851045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:47.851045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:47.851045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:47.851045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:47.851045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:47.851045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 23:31:47.851045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 23:31:47.851045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 23:31:47.851932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 23:31:47.851932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:47.851932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:47.851932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:47.851932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:47.851932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:47.851932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:47.851932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:47.851932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:47.851932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:47.851932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:47.851932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:47.851932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:47.851932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:47.851932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:47.851932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:47.851932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:47.852850 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 23:31:47.852850 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 23:31:47.852850 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 23:31:47.852850 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 23:31:47.852850 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 23:31:47.852850 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 23:31:47.852850 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:47.852850 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:47.855371 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 471/948] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:31:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:47.861674 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d2d7a-7ede-4e9b-b50a-024731e81673 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:47.880112 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:47.972166 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d2d7a-7ede-4e9b-b50a-024731e81673 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:48.006953 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344c5088-0614-4c34-9097-8d026ab87104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:48.007924 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c5088-0614-4c34-9097-8d026ab87104 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:48.008447 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c5088-0614-4c34-9097-8d026ab87104 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:48.008924 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c5088-0614-4c34-9097-8d026ab87104 None admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:48.009814 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344c5088-0614-4c34-9097-8d026ab87104 None admin] RBAC: Adding query filter params (name=service) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:48.010309 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344c5088-0614-4c34-9097-8d026ab87104 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:48.012154 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344c5088-0614-4c34-9097-8d026ab87104 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:48.033127 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 478/949] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Aug 17 23:31:47 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:48.577693 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e02c20-639b-435e-b3c4-ee4f304ccb2c None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:48.578198 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e02c20-639b-435e-b3c4-ee4f304ccb2c None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:48.578300 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e02c20-639b-435e-b3c4-ee4f304ccb2c None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:48.579833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 472/950] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:48.585021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8696e4-b5f9-4ffc-9699-2b232b15f0db None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:48.585317 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8696e4-b5f9-4ffc-9699-2b232b15f0db None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:48.585602 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8696e4-b5f9-4ffc-9699-2b232b15f0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:48.605046 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-ea8696e4-b5f9-4ffc-9699-2b232b15f0db None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:48.606589 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-ea8696e4-b5f9-4ffc-9699-2b232b15f0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:48.677649 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 479/951] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:31:48.945819 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e02c20-639b-435e-b3c4-ee4f304ccb2c None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:49.019177 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:49.055678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142cb16-d9a3-48c1-a2c4-d47091ab9d9b None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:49.056754 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142cb16-d9a3-48c1-a2c4-d47091ab9d9b None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:49.056868 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142cb16-d9a3-48c1-a2c4-d47091ab9d9b None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:49.062027 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 480/952] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:49.063613 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852a797-6157-4a2c-9652-9d2fc059b0ba None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:49.063855 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e02c20-639b-435e-b3c4-ee4f304ccb2c None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:49.064000 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852a797-6157-4a2c-9652-9d2fc059b0ba None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:49.064185 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852a797-6157-4a2c-9652-9d2fc059b0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:49.071380 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:49.086331 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-d852a797-6157-4a2c-9652-9d2fc059b0ba None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:49.088021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-d852a797-6157-4a2c-9652-9d2fc059b0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:49.108952 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78ff7e1-5fc4-4091-8b09-d6029542996a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:49.109641 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ff7e1-5fc4-4091-8b09-d6029542996a None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:49.109984 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ff7e1-5fc4-4091-8b09-d6029542996a None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:49.110260 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ff7e1-5fc4-4091-8b09-d6029542996a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:49.124370 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78ff7e1-5fc4-4091-8b09-d6029542996a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:31:49.129939 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-b78ff7e1-5fc4-4091-8b09-d6029542996a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 17 23:31:49.129939 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:49.129939 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:49.129939 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:49.129939 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:49.129939 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:49.129939 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:49.129939 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:49.129939 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:49.129939 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:49.129939 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:49.129939 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:49.129939 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:31:49.129939 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:31:49.129939 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:31:49.129939 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:31:49.131087 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:31:49.131087 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:31:49.131087 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:31:49.131087 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:31:49.131087 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:49.131087 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:49.131087 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:49.131087 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:49.131087 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:49.131087 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:49.131087 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:49.131087 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:49.131087 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:49.131087 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:49.131087 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:49.131087 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:49.132649 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:49.132649 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:49.132649 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:49.132649 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:49.132649 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:31:49.132649 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:31:49.132649 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:31:49.132649 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:31:49.132649 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:31:49.132649 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:31:49.132649 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 17 23:31:49.132649 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:49.132649 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 473/953] 10.222.0.26 () {56 vars in 1120 bytes} [Wed Aug 17 23:31:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:49.138017 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78ff7e1-5fc4-4091-8b09-d6029542996a None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:49.188497 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 481/954] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 23:31:49.236362 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46913c-a47c-4f07-af18-6037ad881105 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:49.236658 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46913c-a47c-4f07-af18-6037ad881105 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:49.237451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46913c-a47c-4f07-af18-6037ad881105 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:49.239542 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 482/955] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 23:31:49.244501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b19688-7af1-4760-92bb-6e16ce3ec9ac None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:49.244907 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b19688-7af1-4760-92bb-6e16ce3ec9ac None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:49.245551 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b19688-7af1-4760-92bb-6e16ce3ec9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:49.257532 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78ff7e1-5fc4-4091-8b09-d6029542996a None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:49.271641 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-60b19688-7af1-4760-92bb-6e16ce3ec9ac None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:49.272852 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-60b19688-7af1-4760-92bb-6e16ce3ec9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:49.285881 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:49.300773 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b303db-d16d-4dca-a02d-c428df6b8fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:49.301675 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b303db-d16d-4dca-a02d-c428df6b8fac None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:49.302285 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b303db-d16d-4dca-a02d-c428df6b8fac None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:49.303059 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b303db-d16d-4dca-a02d-c428df6b8fac None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:49.304215 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b303db-d16d-4dca-a02d-c428df6b8fac None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:49.304726 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b303db-d16d-4dca-a02d-c428df6b8fac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:49.307059 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b303db-d16d-4dca-a02d-c428df6b8fac None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:49.317545 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 474/956] 10.222.0.26 () {56 vars in 1115 bytes} [Wed Aug 17 23:31:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:49.324156 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b303db-d16d-4dca-a02d-c428df6b8fac None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:49.356537 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 483/957] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:31:49.437481 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b303db-d16d-4dca-a02d-c428df6b8fac None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:49.473145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc047dd-ed51-4487-9c91-9e7e2c8a41c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:49.473782 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc047dd-ed51-4487-9c91-9e7e2c8a41c3 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:49.474061 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc047dd-ed51-4487-9c91-9e7e2c8a41c3 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:49.474337 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc047dd-ed51-4487-9c91-9e7e2c8a41c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:49.481297 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b19688-7af1-4760-92bb-6e16ce3ec9ac None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:49.486539 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc047dd-ed51-4487-9c91-9e7e2c8a41c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:49.488021 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc047dd-ed51-4487-9c91-9e7e2c8a41c3 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:49.506536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-5dc047dd-ed51-4487-9c91-9e7e2c8a41c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:49.506536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:49.506536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:49.506536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:49.506536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:49.506536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:49.506536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:49.506536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:49.506536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:49.506536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:49.506536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:49.506536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:49.506536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:49.506536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:49.506536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:49.506536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:49.507524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:49.507524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:49.507524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:49.507524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:49.507524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:49.507524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:49.507524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:49.507524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:49.507524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:49.507524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:49.507524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:49.507524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:49.507524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:49.507524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:49.507524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:49.507524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:49.507524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:49.508475 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:49.508475 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:49.508475 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:49.508475 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 475/958] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:31:49.514219 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc047dd-ed51-4487-9c91-9e7e2c8a41c3 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:49.561023 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:49.565500 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:49.605277 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b19688-7af1-4760-92bb-6e16ce3ec9ac None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:49.629078 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc047dd-ed51-4487-9c91-9e7e2c8a41c3 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:49.645022 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b790f55-83c3-47d9-829a-36816c8d6e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:49.645658 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b790f55-83c3-47d9-829a-36816c8d6e0d None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:49.645938 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b790f55-83c3-47d9-829a-36816c8d6e0d None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:49.646211 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b790f55-83c3-47d9-829a-36816c8d6e0d None admin] PATH_INFO: `/v3/services/compute` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:49.646787 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b790f55-83c3-47d9-829a-36816c8d6e0d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:49.648222 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b790f55-83c3-47d9-829a-36816c8d6e0d None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:49.657775 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-0b790f55-83c3-47d9-829a-36816c8d6e0d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 17 23:31:49.657775 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:49.657775 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 23:31:49.657775 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 23:31:49.657775 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 23:31:49.657775 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 23:31:49.657775 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 23:31:49.657775 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:49.657775 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 17 23:31:49.657775 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:49.657775 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 23:31:49.657775 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:49.657775 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:49.657775 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:49.657775 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:49.657775 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:49.657775 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:49.658899 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:49.658899 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:49.658899 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:49.658899 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:49.658899 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:49.658899 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:49.658899 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 23:31:49.658899 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 23:31:49.658899 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 23:31:49.658899 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 23:31:49.658899 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:49.658899 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:49.658899 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:49.658899 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:49.658899 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:49.658899 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:49.659751 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:49.659751 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:49.659751 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:49.659751 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:49.659751 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:49.659751 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:49.659751 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:49.659751 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:49.659751 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:49.659751 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:49.659751 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 23:31:49.659751 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:49.659751 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 17 23:31:49.659751 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:49.663524 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 484/959] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:31:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:49.664463 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b790f55-83c3-47d9-829a-36816c8d6e0d None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:49.670268 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2cbad7-a32d-46e6-b183-94575f8e6af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:49.670881 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2cbad7-a32d-46e6-b183-94575f8e6af9 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:49.671163 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2cbad7-a32d-46e6-b183-94575f8e6af9 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:49.671437 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2cbad7-a32d-46e6-b183-94575f8e6af9 None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:49.672125 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2cbad7-a32d-46e6-b183-94575f8e6af9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:49.672432 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2cbad7-a32d-46e6-b183-94575f8e6af9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:49.673806 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2cbad7-a32d-46e6-b183-94575f8e6af9 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:49.687022 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 476/960] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:31:49.687690 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2cbad7-a32d-46e6-b183-94575f8e6af9 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:49.779693 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b790f55-83c3-47d9-829a-36816c8d6e0d None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:49.795162 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:49.799833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2cbad7-a32d-46e6-b183-94575f8e6af9 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:49.807133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:49.815727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fac905-88a0-4335-b26e-9e5b3d64d2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:49.816396 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fac905-88a0-4335-b26e-9e5b3d64d2ee None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:49.816678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fac905-88a0-4335-b26e-9e5b3d64d2ee None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:49.816947 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fac905-88a0-4335-b26e-9e5b3d64d2ee None admin] PATH_INFO: `/v3/services/object-store` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:49.817606 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fac905-88a0-4335-b26e-9e5b3d64d2ee None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:49.818994 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fac905-88a0-4335-b26e-9e5b3d64d2ee None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:49.828405 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-b1fac905-88a0-4335-b26e-9e5b3d64d2ee None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 23:31:49.828405 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:49.828405 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 23:31:49.828405 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 23:31:49.828405 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 23:31:49.828405 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 23:31:49.828405 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 23:31:49.828405 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:49.828405 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 23:31:49.828405 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:49.828405 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 23:31:49.828405 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:49.828405 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:49.828405 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:49.828405 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:49.828405 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:49.828405 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:49.829429 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:49.829429 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:49.829429 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:49.829429 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:49.829429 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:49.829429 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:49.829429 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 23:31:49.829429 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 23:31:49.829429 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 23:31:49.829429 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 23:31:49.829429 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:49.829429 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:49.829429 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:49.829429 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:49.829429 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:49.829429 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:49.830273 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:49.830273 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:49.830273 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:49.830273 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:49.830273 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:49.830273 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:49.830273 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:49.830273 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:49.830273 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:49.830273 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:49.830273 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 23:31:49.830273 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:49.830273 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 23:31:49.830273 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:49.830273 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 485/961] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Aug 17 23:31:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:49.831079 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fac905-88a0-4335-b26e-9e5b3d64d2ee None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:49.843591 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba2da7-a2c2-4f12-8f10-5fd3fc39e7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:49.844550 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba2da7-a2c2-4f12-8f10-5fd3fc39e7f2 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:49.844994 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba2da7-a2c2-4f12-8f10-5fd3fc39e7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:49.845274 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba2da7-a2c2-4f12-8f10-5fd3fc39e7f2 None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:49.845914 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba2da7-a2c2-4f12-8f10-5fd3fc39e7f2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:49.846218 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba2da7-a2c2-4f12-8f10-5fd3fc39e7f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:49.847585 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba2da7-a2c2-4f12-8f10-5fd3fc39e7f2 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:49.853258 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 477/962] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Aug 17 23:31:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:49.854051 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba2da7-a2c2-4f12-8f10-5fd3fc39e7f2 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:49.961804 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fac905-88a0-4335-b26e-9e5b3d64d2ee None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:49.974123 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba2da7-a2c2-4f12-8f10-5fd3fc39e7f2 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:49.997529 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0caf1-cb5a-412e-a88e-244843882715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:49.998164 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0caf1-cb5a-412e-a88e-244843882715 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:49.998444 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0caf1-cb5a-412e-a88e-244843882715 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:49.998720 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0caf1-cb5a-412e-a88e-244843882715 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:50.006275 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f0caf1-cb5a-412e-a88e-244843882715 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:50.007419 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d46e32c-83d3-4ec3-8984-e52ef44010a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:50.007718 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f0caf1-cb5a-412e-a88e-244843882715 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:50.008055 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46e32c-83d3-4ec3-8984-e52ef44010a1 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:50.008337 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46e32c-83d3-4ec3-8984-e52ef44010a1 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:50.008607 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46e32c-83d3-4ec3-8984-e52ef44010a1 None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:50.009765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d46e32c-83d3-4ec3-8984-e52ef44010a1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:50.010072 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d46e32c-83d3-4ec3-8984-e52ef44010a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:50.011457 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d46e32c-83d3-4ec3-8984-e52ef44010a1 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:50.020717 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 478/963] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Aug 17 23:31:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:50.020717 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d46e32c-83d3-4ec3-8984-e52ef44010a1 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:50.020958 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-02f0caf1-cb5a-412e-a88e-244843882715 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 17 23:31:50.020958 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:50.020958 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:50.020958 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:50.020958 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:50.020958 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:50.020958 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:50.020958 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:50.020958 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:50.020958 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:50.020958 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:50.020958 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:50.020958 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 23:31:50.020958 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 23:31:50.020958 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 23:31:50.020958 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 23:31:50.023300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:50.023300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:50.023300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 23:31:50.023300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:50.023300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 23:31:50.023300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:50.023300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:50.023300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:50.023300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:50.023300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:50.023300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:50.023300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:50.023300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:50.023300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:50.023300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:50.023300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:50.023300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:50.023300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:50.024302 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:50.024302 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:50.024302 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 23:31:50.024302 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 23:31:50.024302 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 23:31:50.024302 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 23:31:50.024302 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 23:31:50.024302 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 23:31:50.024302 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 17 23:31:50.024302 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:50.024302 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 486/964] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Aug 17 23:31:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:50.024302 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0caf1-cb5a-412e-a88e-244843882715 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:50.055682 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:50.079265 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:50.128444 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d46e32c-83d3-4ec3-8984-e52ef44010a1 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:50.128641 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0caf1-cb5a-412e-a88e-244843882715 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:50.160907 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e517c5a3-5abb-4421-8d45-144f2d55a4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:50.161381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f564cc3-29ac-4084-8df9-2808bdf566f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:50.161504 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517c5a3-5abb-4421-8d45-144f2d55a4b4 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:50.161768 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517c5a3-5abb-4421-8d45-144f2d55a4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:50.161976 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f564cc3-29ac-4084-8df9-2808bdf566f4 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:50.162079 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517c5a3-5abb-4421-8d45-144f2d55a4b4 None admin] PATH_INFO: `/v3/services` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:50.162254 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f564cc3-29ac-4084-8df9-2808bdf566f4 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:50.162524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f564cc3-29ac-4084-8df9-2808bdf566f4 None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:50.162622 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e517c5a3-5abb-4421-8d45-144f2d55a4b4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:50.162926 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e517c5a3-5abb-4421-8d45-144f2d55a4b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:50.163145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f564cc3-29ac-4084-8df9-2808bdf566f4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:50.163443 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f564cc3-29ac-4084-8df9-2808bdf566f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:50.164339 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e517c5a3-5abb-4421-8d45-144f2d55a4b4 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:50.164834 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f564cc3-29ac-4084-8df9-2808bdf566f4 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:50.169183 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 479/965] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Aug 17 23:31:50 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:50.169359 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 487/966] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Aug 17 23:31:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:50.169958 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f564cc3-29ac-4084-8df9-2808bdf566f4 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:50.170813 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadab30f-2560-4347-977e-be6a8d286ade None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:50.171100 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadab30f-2560-4347-977e-be6a8d286ade None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:50.171376 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadab30f-2560-4347-977e-be6a8d286ade None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:50.172869 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 488/967] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:50.177762 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadab30f-2560-4347-977e-be6a8d286ade None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:50.287312 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:50.291813 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadab30f-2560-4347-977e-be6a8d286ade None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:50.299842 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f564cc3-29ac-4084-8df9-2808bdf566f4 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:50.311014 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:50.331137 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958768c-1c83-4688-9b78-b9c8b87bbbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:50.331754 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958768c-1c83-4688-9b78-b9c8b87bbbc5 None admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:50.332063 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958768c-1c83-4688-9b78-b9c8b87bbbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:50.332338 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958768c-1c83-4688-9b78-b9c8b87bbbc5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:50.332998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7958768c-1c83-4688-9b78-b9c8b87bbbc5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:50.334358 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7958768c-1c83-4688-9b78-b9c8b87bbbc5 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:50.339924 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcce968-8d37-449b-b320-49857c1d2089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:50.340572 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcce968-8d37-449b-b320-49857c1d2089 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:50.340855 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcce968-8d37-449b-b320-49857c1d2089 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:50.341125 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcce968-8d37-449b-b320-49857c1d2089 None admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:50.341865 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcce968-8d37-449b-b320-49857c1d2089 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:50.342199 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcce968-8d37-449b-b320-49857c1d2089 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:50.343593 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcce968-8d37-449b-b320-49857c1d2089 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:50.355910 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 489/968] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:31:50 2022] POST /identity/v3/endpoints => generated 349 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:31:50.357270 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc2084-c96c-4655-9450-abd3406ff1a1 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:50.357561 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc2084-c96c-4655-9450-abd3406ff1a1 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:50.357836 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc2084-c96c-4655-9450-abd3406ff1a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:50.372426 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 480/969] 10.222.0.26 () {56 vars in 1149 bytes} [Wed Aug 17 23:31:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:50.378247 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcce968-8d37-449b-b320-49857c1d2089 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:50.390077 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-31cc2084-c96c-4655-9450-abd3406ff1a1 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:50.392432 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-31cc2084-c96c-4655-9450-abd3406ff1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:50.491381 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 490/970] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:31:50.507656 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcce968-8d37-449b-b320-49857c1d2089 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:50.545030 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502ffe4-91fd-4ef6-836b-41be60ed1039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:50.545998 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502ffe4-91fd-4ef6-836b-41be60ed1039 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:50.546425 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502ffe4-91fd-4ef6-836b-41be60ed1039 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:50.546694 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502ffe4-91fd-4ef6-836b-41be60ed1039 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:50.565185 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:50.618483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b502ffe4-91fd-4ef6-836b-41be60ed1039 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:50.619937 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b502ffe4-91fd-4ef6-836b-41be60ed1039 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:50.724194 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 481/971] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:31:50 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 23:31:50.733370 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc2084-c96c-4655-9450-abd3406ff1a1 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:50.780719 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502ffe4-91fd-4ef6-836b-41be60ed1039 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:50.838182 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:50.839066 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:50.844514 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc2084-c96c-4655-9450-abd3406ff1a1 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:50.893090 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d1c59-65a7-449b-ba85-0f319d86ea60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:50.894011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d1c59-65a7-449b-ba85-0f319d86ea60 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:50.894319 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d1c59-65a7-449b-ba85-0f319d86ea60 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:50.894975 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d1c59-65a7-449b-ba85-0f319d86ea60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:50.897655 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502ffe4-91fd-4ef6-836b-41be60ed1039 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:50.908667 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d1c59-65a7-449b-ba85-0f319d86ea60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:50.910813 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d1c59-65a7-449b-ba85-0f319d86ea60 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:50.927761 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-273d1c59-65a7-449b-ba85-0f319d86ea60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:50.927761 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:50.927761 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:50.927761 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:50.927761 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:50.927761 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:50.927761 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:50.927761 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:50.927761 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:50.927761 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:50.927761 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:50.927761 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:50.927761 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:31:50.927761 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:31:50.927761 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:31:50.927761 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:31:50.928914 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:50.928914 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:50.928914 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:50.928914 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:50.928914 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:50.928914 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:50.928914 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:50.928914 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:50.928914 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:50.928914 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:50.928914 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:50.928914 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:50.928914 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:50.928914 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:50.928914 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:50.928914 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:50.928914 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:31:50.929810 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:31:50.929810 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:31:50.929810 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:50.933595 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 491/972] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:50.934853 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d1c59-65a7-449b-ba85-0f319d86ea60 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:50.941522 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd5b6c5-1b35-46ef-b1f7-96cdcd57289b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:50.942137 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5b6c5-1b35-46ef-b1f7-96cdcd57289b None admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:50.942414 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5b6c5-1b35-46ef-b1f7-96cdcd57289b None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:50.942693 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5b6c5-1b35-46ef-b1f7-96cdcd57289b None admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:50.943346 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd5b6c5-1b35-46ef-b1f7-96cdcd57289b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:50.945398 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd5b6c5-1b35-46ef-b1f7-96cdcd57289b None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:50.949192 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: WARNING keystone.server.flask.common [None req-dbd5b6c5-1b35-46ef-b1f7-96cdcd57289b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 17 23:31:50.966010 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.sql.core [None req-dbd5b6c5-1b35-46ef-b1f7-96cdcd57289b None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 17 23:31:50.966010 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 17 23:31:50.966010 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [parameters: {'user_id': '2c10845644c64ca9a1dbcb6daacb336a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 17 23:31:50.966010 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=85835) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 17 23:31:50.977381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-dbd5b6c5-1b35-46ef-b1f7-96cdcd57289b None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 17 23:31:50.977381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:50.977381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 17 23:31:50.977381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 17 23:31:50.977381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 17 23:31:50.977381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 17 23:31:50.977381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 17 23:31:50.977381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application result = self._query(query) Aug 17 23:31:50.977381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 17 23:31:50.977381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application conn.query(q) Aug 17 23:31:50.977381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 17 23:31:50.977381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 17 23:31:50.977381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 17 23:31:50.977381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application result.read() Aug 17 23:31:50.977381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 17 23:31:50.979694 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 17 23:31:50.979694 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 17 23:31:50.979694 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 17 23:31:50.979694 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 17 23:31:50.979694 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 17 23:31:50.979694 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 17 23:31:50.979694 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 17 23:31:50.979694 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 17 23:31:50.979694 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:50.979694 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 17 23:31:50.979694 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:50.979694 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:50.979694 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 17 23:31:50.979694 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 17 23:31:50.979694 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 17 23:31:50.979694 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 17 23:31:50.979694 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 17 23:31:50.979694 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application next(self.gen) Aug 17 23:31:50.981213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 17 23:31:50.981213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application yield resource Aug 17 23:31:50.981213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 17 23:31:50.981213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application next(self.gen) Aug 17 23:31:50.981213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 17 23:31:50.981213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self.session.rollback() Aug 17 23:31:50.981213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 17 23:31:50.981213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self.force_reraise() Aug 17 23:31:50.981213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 17 23:31:50.981213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise self.value Aug 17 23:31:50.981213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 17 23:31:50.981213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 17 23:31:50.981213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 17 23:31:50.981213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application session.commit() Aug 17 23:31:50.981213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 17 23:31:50.981213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 17 23:31:50.981213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 17 23:31:50.982302 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self._prepare_impl() Aug 17 23:31:50.982302 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 17 23:31:50.982302 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self.session.flush() Aug 17 23:31:50.982302 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 17 23:31:50.982302 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self._flush(objects) Aug 17 23:31:50.982302 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 17 23:31:50.982302 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 17 23:31:50.982302 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 17 23:31:50.982302 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application compat.raise_( Aug 17 23:31:50.982302 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 17 23:31:50.982302 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception Aug 17 23:31:50.982302 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 17 23:31:50.982302 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application flush_context.execute() Aug 17 23:31:50.982302 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 17 23:31:50.982302 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rec.execute(self) Aug 17 23:31:50.982302 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 17 23:31:50.982302 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 17 23:31:50.983457 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 17 23:31:50.983457 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 17 23:31:50.983457 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 17 23:31:50.983457 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 17 23:31:50.983457 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 17 23:31:50.983457 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 17 23:31:50.983457 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 17 23:31:50.983457 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 17 23:31:50.983457 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 17 23:31:50.983457 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 17 23:31:50.983457 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 17 23:31:50.983457 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 17 23:31:50.983457 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 17 23:31:50.983457 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 17 23:31:50.983457 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 17 23:31:50.983457 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception Aug 17 23:31:50.985029 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 17 23:31:50.985029 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 17 23:31:50.985029 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 17 23:31:50.985029 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 17 23:31:50.985029 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 17 23:31:50.985029 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application result = self._query(query) Aug 17 23:31:50.985029 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 17 23:31:50.985029 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application conn.query(q) Aug 17 23:31:50.985029 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 17 23:31:50.985029 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 17 23:31:50.985029 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 17 23:31:50.985029 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application result.read() Aug 17 23:31:50.985029 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 17 23:31:50.985029 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 17 23:31:50.985029 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 17 23:31:50.985029 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 17 23:31:50.986128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 17 23:31:50.986128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 17 23:31:50.986128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 17 23:31:50.986128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 17 23:31:50.986128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 17 23:31:50.986128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 17 23:31:50.986128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [parameters: {'user_id': '2c10845644c64ca9a1dbcb6daacb336a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 17 23:31:50.986128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 17 23:31:50.986128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:50.986128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 23:31:50.986128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:50.986128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:50.986128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:50.986128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:50.986128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:50.987105 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:50.987105 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:50.987105 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:50.987105 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:50.987105 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:50.987105 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:50.987105 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:50.987105 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 17 23:31:50.987105 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 17 23:31:50.987105 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:50.987105 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:50.987105 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 23:31:50.987105 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:50.987105 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 23:31:50.987105 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:50.987105 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 17 23:31:50.987105 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 17 23:31:51.000934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 17 23:31:51.000934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 17 23:31:51.000934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 17 23:31:51.000934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 17 23:31:51.000934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 17 23:31:51.000934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:51.000934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 482/973] 10.222.0.26 () {60 vars in 1131 bytes} [Wed Aug 17 23:31:50 2022] POST /identity/v3/users => generated 160 bytes in 199 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:31:51.000934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd5b6c5-1b35-46ef-b1f7-96cdcd57289b None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:51.052687 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d1c59-65a7-449b-ba85-0f319d86ea60 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:51.093592 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b12eeb-d6a0-42a1-bc50-224ddd1052ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:51.094681 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b12eeb-d6a0-42a1-bc50-224ddd1052ee None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:51.094970 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b12eeb-d6a0-42a1-bc50-224ddd1052ee None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:51.095249 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b12eeb-d6a0-42a1-bc50-224ddd1052ee None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:51.095913 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b12eeb-d6a0-42a1-bc50-224ddd1052ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:51.096247 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b12eeb-d6a0-42a1-bc50-224ddd1052ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:51.098512 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b12eeb-d6a0-42a1-bc50-224ddd1052ee None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:51.115985 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 492/974] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:31:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:31:51.121724 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b12eeb-d6a0-42a1-bc50-224ddd1052ee None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:51.125994 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd5b6c5-1b35-46ef-b1f7-96cdcd57289b None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:51.137404 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:51.138935 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:51.171447 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317fca5-88fb-4b54-98cd-7ba646c947eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:51.172513 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317fca5-88fb-4b54-98cd-7ba646c947eb None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:51.173785 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317fca5-88fb-4b54-98cd-7ba646c947eb None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:51.174136 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317fca5-88fb-4b54-98cd-7ba646c947eb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:51.183709 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5317fca5-88fb-4b54-98cd-7ba646c947eb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:51.186253 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5317fca5-88fb-4b54-98cd-7ba646c947eb None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:51.202950 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-5317fca5-88fb-4b54-98cd-7ba646c947eb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 17 23:31:51.202950 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:51.202950 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:51.202950 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:51.202950 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:51.202950 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:51.202950 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:51.202950 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:51.202950 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:51.202950 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:51.202950 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:51.202950 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:51.202950 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 23:31:51.202950 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 23:31:51.202950 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 23:31:51.202950 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 23:31:51.204875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:51.204875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:51.204875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 23:31:51.204875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:51.204875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 23:31:51.204875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:31:51.204875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:51.204875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:51.204875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:51.204875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:51.204875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:51.204875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:51.204875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:51.204875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:51.204875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:51.204875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:51.204875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:51.204875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:51.206045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:51.206045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:51.206045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 23:31:51.206045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 23:31:51.206045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 23:31:51.206045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 23:31:51.206045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 23:31:51.206045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 23:31:51.206045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 17 23:31:51.206045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:51.211643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 483/975] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:31:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:31:51.211643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317fca5-88fb-4b54-98cd-7ba646c947eb None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:51.259013 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b12eeb-d6a0-42a1-bc50-224ddd1052ee None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:51.307375 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b076515-715d-4994-b47a-7cf91b9a2db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:51.308221 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b076515-715d-4994-b47a-7cf91b9a2db3 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:51.308770 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b076515-715d-4994-b47a-7cf91b9a2db3 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:51.309147 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b076515-715d-4994-b47a-7cf91b9a2db3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:51.318338 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317fca5-88fb-4b54-98cd-7ba646c947eb None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:51.318550 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b076515-715d-4994-b47a-7cf91b9a2db3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:51.320270 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b076515-715d-4994-b47a-7cf91b9a2db3 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:51.335415 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-4b076515-715d-4994-b47a-7cf91b9a2db3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:51.335415 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:51.335415 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:51.335415 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:51.335415 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:51.335415 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:51.335415 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:51.335415 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:51.335415 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:51.335415 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:51.335415 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:51.335415 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:51.335415 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 23:31:51.335415 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 23:31:51.335415 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 23:31:51.337132 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 23:31:51.337132 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:51.337132 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:51.337132 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:51.337132 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:51.337132 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:51.337132 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:51.337132 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:51.337132 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:51.337132 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:51.337132 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:51.337132 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:51.337132 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:51.337132 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:51.337132 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:51.337132 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:51.337132 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:51.339727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 23:31:51.339727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 23:31:51.339727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 23:31:51.339727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 23:31:51.339727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 23:31:51.339727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 23:31:51.339727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:31:51.339727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:51.339727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 493/976] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:31:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:51.345595 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b076515-715d-4994-b47a-7cf91b9a2db3 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:51.361341 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc61a5-6bfa-45b9-a5be-0a3f14da319c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:51.362336 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc61a5-6bfa-45b9-a5be-0a3f14da319c None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:51.362619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc61a5-6bfa-45b9-a5be-0a3f14da319c None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:51.362894 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc61a5-6bfa-45b9-a5be-0a3f14da319c None admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:51.363734 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc61a5-6bfa-45b9-a5be-0a3f14da319c None admin] RBAC: Adding query filter params (name=cinder) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:51.364182 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc61a5-6bfa-45b9-a5be-0a3f14da319c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:51.365752 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc61a5-6bfa-45b9-a5be-0a3f14da319c None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:51.411164 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 484/977] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Aug 17 23:31:51 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:31:51.430747 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:51.473903 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b076515-715d-4994-b47a-7cf91b9a2db3 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:51.511833 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce0c01b-2d5e-4ff7-b9b0-1f59950bafcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:51.512552 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0c01b-2d5e-4ff7-b9b0-1f59950bafcc None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:51.512832 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0c01b-2d5e-4ff7-b9b0-1f59950bafcc None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:51.513104 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0c01b-2d5e-4ff7-b9b0-1f59950bafcc None admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:51.513881 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce0c01b-2d5e-4ff7-b9b0-1f59950bafcc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:51.514190 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce0c01b-2d5e-4ff7-b9b0-1f59950bafcc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:51.515661 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce0c01b-2d5e-4ff7-b9b0-1f59950bafcc None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:51.534550 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 494/978] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 17 23:31:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:31:51.540941 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc61a5-6bfa-45b9-a5be-0a3f14da319c None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:51.577934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:51.658080 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc61a5-6bfa-45b9-a5be-0a3f14da319c None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:51.692985 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66997d7b-a854-4687-a460-5262a300cdbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:51.693590 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66997d7b-a854-4687-a460-5262a300cdbc None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:51.693866 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66997d7b-a854-4687-a460-5262a300cdbc None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:51.694134 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66997d7b-a854-4687-a460-5262a300cdbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:51.694867 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66997d7b-a854-4687-a460-5262a300cdbc None admin] RBAC: Adding query filter params (role.id=dc16b44096d040dfb737c287986be85e, scope.project.id=4de86915ef4340248ba8a8f51d56594e, user.id=d0a13dbffb4a49508b7fb661cf84e803) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:51.695169 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66997d7b-a854-4687-a460-5262a300cdbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:51.696707 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66997d7b-a854-4687-a460-5262a300cdbc None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:51.703038 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 485/979] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:31:51 2022] GET /identity/v3/role_assignments?user.id=d0a13dbffb4a49508b7fb661cf84e803&scope.project.id=4de86915ef4340248ba8a8f51d56594e&role.id=dc16b44096d040dfb737c287986be85e => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:52.132067 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f46da09-3dc9-4a0e-8855-21ec15a1d2d4 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:52.132580 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f46da09-3dc9-4a0e-8855-21ec15a1d2d4 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:52.132658 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f46da09-3dc9-4a0e-8855-21ec15a1d2d4 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:52.134593 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 495/980] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:52.139608 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd5ebd5-ee7b-4bed-b182-3aad8fa3998a None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:52.140150 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd5ebd5-ee7b-4bed-b182-3aad8fa3998a None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:52.140150 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd5ebd5-ee7b-4bed-b182-3aad8fa3998a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:52.158920 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-fdd5ebd5-ee7b-4bed-b182-3aad8fa3998a None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:52.160447 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-fdd5ebd5-ee7b-4bed-b182-3aad8fa3998a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:52.225549 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 486/981] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:31:52.389726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e67a9-a152-47f0-8599-327d73bf419f None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:52.390057 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e67a9-a152-47f0-8599-327d73bf419f None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:52.390233 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e67a9-a152-47f0-8599-327d73bf419f None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:52.391652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 496/982] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:52.396636 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4a110-9bfc-448b-b717-9d539c14e690 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:52.396918 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4a110-9bfc-448b-b717-9d539c14e690 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:52.397185 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4a110-9bfc-448b-b717-9d539c14e690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:52.414918 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-57c4a110-9bfc-448b-b717-9d539c14e690 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:52.416156 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-57c4a110-9bfc-448b-b717-9d539c14e690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:52.432431 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:52.489541 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 487/983] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:31:52.495805 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30e67a9-a152-47f0-8599-327d73bf419f None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:52.599012 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30e67a9-a152-47f0-8599-327d73bf419f None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:52.627045 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:52.633948 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96376a80-7a9d-4520-a1cb-f9deab04ac56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:52.635211 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96376a80-7a9d-4520-a1cb-f9deab04ac56 None admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:52.635492 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96376a80-7a9d-4520-a1cb-f9deab04ac56 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:52.635761 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96376a80-7a9d-4520-a1cb-f9deab04ac56 None admin] PATH_INFO: `/v3/services` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:52.636436 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96376a80-7a9d-4520-a1cb-f9deab04ac56 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:52.637843 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96376a80-7a9d-4520-a1cb-f9deab04ac56 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:52.652968 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 497/984] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Aug 17 23:31:52 2022] POST /identity/v3/services => generated 242 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:31:52.759477 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4a110-9bfc-448b-b717-9d539c14e690 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:52.864808 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4a110-9bfc-448b-b717-9d539c14e690 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:52.908270 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd63d5df-3c04-4a82-acfc-c240418413a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:52.908826 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd63d5df-3c04-4a82-acfc-c240418413a0 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:52.909107 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd63d5df-3c04-4a82-acfc-c240418413a0 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:52.909378 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd63d5df-3c04-4a82-acfc-c240418413a0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:52.909963 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd63d5df-3c04-4a82-acfc-c240418413a0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:52.911332 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd63d5df-3c04-4a82-acfc-c240418413a0 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:52.920765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-bd63d5df-3c04-4a82-acfc-c240418413a0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 23:31:52.920765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:52.920765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 23:31:52.920765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 23:31:52.920765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 23:31:52.920765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 23:31:52.920765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 23:31:52.920765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:52.920765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 23:31:52.920765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:52.920765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 23:31:52.920765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:52.920765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:52.920765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:52.920765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:52.920765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:52.920765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:52.921742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:52.921742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:52.921742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:52.921742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:52.921742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:52.921742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:52.921742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 23:31:52.921742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 23:31:52.921742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 23:31:52.921742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 23:31:52.921742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:52.921742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:52.921742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:52.921742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:52.921742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:52.921742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:52.922611 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:52.922611 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:52.922611 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:52.922611 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:52.922611 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:52.922611 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:52.922611 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:52.922611 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:52.922611 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:52.922611 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:52.922611 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 23:31:52.922611 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:52.922611 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 23:31:52.922611 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:52.922611 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 488/985] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Aug 17 23:31:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:52.929399 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96376a80-7a9d-4520-a1cb-f9deab04ac56 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:53.031720 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96376a80-7a9d-4520-a1cb-f9deab04ac56 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:53.065864 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f9121-cea9-4c61-a6f0-646abaa54b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:53.066472 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f9121-cea9-4c61-a6f0-646abaa54b21 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:53.066750 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f9121-cea9-4c61-a6f0-646abaa54b21 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:53.067020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f9121-cea9-4c61-a6f0-646abaa54b21 None admin] PATH_INFO: `/v3/services` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:53.067643 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1f9121-cea9-4c61-a6f0-646abaa54b21 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:53.067944 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1f9121-cea9-4c61-a6f0-646abaa54b21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:53.069334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1f9121-cea9-4c61-a6f0-646abaa54b21 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:53.074429 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 498/986] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Aug 17 23:31:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:53.079891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd63d5df-3c04-4a82-acfc-c240418413a0 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:53.095271 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:53.185057 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd63d5df-3c04-4a82-acfc-c240418413a0 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:53.218973 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20f5b6-1984-41c6-9e30-29ad31513a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:53.219577 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20f5b6-1984-41c6-9e30-29ad31513a9f None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:53.219855 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20f5b6-1984-41c6-9e30-29ad31513a9f None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:53.220152 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20f5b6-1984-41c6-9e30-29ad31513a9f None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:53.220780 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20f5b6-1984-41c6-9e30-29ad31513a9f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:53.221084 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20f5b6-1984-41c6-9e30-29ad31513a9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:53.222451 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20f5b6-1984-41c6-9e30-29ad31513a9f None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:53.227524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 489/987] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Aug 17 23:31:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:53.233727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f9121-cea9-4c61-a6f0-646abaa54b21 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:53.289097 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:53.336326 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f9121-cea9-4c61-a6f0-646abaa54b21 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:53.369106 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ee681-999b-4301-8e16-a01d317a5151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:53.369707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ee681-999b-4301-8e16-a01d317a5151 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:53.369986 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ee681-999b-4301-8e16-a01d317a5151 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:53.370260 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ee681-999b-4301-8e16-a01d317a5151 None admin] PATH_INFO: `/v3/endpoints` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:53.370962 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840ee681-999b-4301-8e16-a01d317a5151 None admin] RBAC: Adding query filter params (interface=admin, service_id=6270541d709148648c5e544ed6d276ac, region_id=RegionOne) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:53.371272 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840ee681-999b-4301-8e16-a01d317a5151 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:53.372667 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840ee681-999b-4301-8e16-a01d317a5151 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:53.377779 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 499/988] 10.222.0.26 () {56 vars in 1249 bytes} [Wed Aug 17 23:31:53 2022] GET /identity/v3/endpoints?service_id=6270541d709148648c5e544ed6d276ac&interface=admin®ion_id=RegionOne => generated 193 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:53.383111 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20f5b6-1984-41c6-9e30-29ad31513a9f None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:53.469643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:53.495697 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20f5b6-1984-41c6-9e30-29ad31513a9f None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:53.530103 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a9140f-a680-4611-b5f0-beef339cddb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:53.530717 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9140f-a680-4611-b5f0-beef339cddb9 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:53.530996 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9140f-a680-4611-b5f0-beef339cddb9 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:53.531265 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9140f-a680-4611-b5f0-beef339cddb9 None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:53.531867 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a9140f-a680-4611-b5f0-beef339cddb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:53.533281 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a9140f-a680-4611-b5f0-beef339cddb9 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:53.539734 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 490/989] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Aug 17 23:31:53 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:31:53.628782 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5031e-1a4e-4929-a809-a1a1c189a3a5 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:53.629124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5031e-1a4e-4929-a809-a1a1c189a3a5 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:53.629336 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5031e-1a4e-4929-a809-a1a1c189a3a5 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:53.630856 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 500/990] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:53.636280 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c1b03-1fee-4473-a0d8-98a3d741c3ed None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:53.636586 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c1b03-1fee-4473-a0d8-98a3d741c3ed None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:53.636832 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c1b03-1fee-4473-a0d8-98a3d741c3ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:53.656606 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-2e4c1b03-1fee-4473-a0d8-98a3d741c3ed None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:53.657828 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-2e4c1b03-1fee-4473-a0d8-98a3d741c3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:53.732651 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 491/991] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:53 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:31:53.999048 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5031e-1a4e-4929-a809-a1a1c189a3a5 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:54.101018 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5031e-1a4e-4929-a809-a1a1c189a3a5 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:54.117438 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:54.136099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e10446-f681-48fa-aebb-3da9402aabdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:54.136804 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e10446-f681-48fa-aebb-3da9402aabdd None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:54.137083 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e10446-f681-48fa-aebb-3da9402aabdd None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:54.137351 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e10446-f681-48fa-aebb-3da9402aabdd None admin] PATH_INFO: `/v3/services/image` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:54.137929 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e10446-f681-48fa-aebb-3da9402aabdd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:54.139286 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e10446-f681-48fa-aebb-3da9402aabdd None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:54.148067 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-09e10446-f681-48fa-aebb-3da9402aabdd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 17 23:31:54.148067 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:54.148067 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 23:31:54.148067 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 23:31:54.148067 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 23:31:54.148067 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 23:31:54.148067 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 23:31:54.148067 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:54.148067 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 17 23:31:54.148067 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:54.148067 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 23:31:54.148067 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:54.148067 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:54.148067 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:54.148067 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:54.148067 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:54.148067 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:54.149099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:54.149099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:54.149099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:54.149099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:54.149099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:54.149099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:54.149099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 23:31:54.149099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 23:31:54.149099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 23:31:54.149099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 23:31:54.149099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:54.149099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:54.149099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:54.149099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:54.149099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:54.149099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:54.149973 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:54.149973 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:54.149973 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:54.149973 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:54.149973 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:54.149973 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:54.149973 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:54.149973 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:54.149973 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:54.149973 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:54.149973 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 23:31:54.149973 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:54.149973 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 17 23:31:54.149973 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:54.152281 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 501/992] 10.222.0.26 () {56 vars in 1105 bytes} [Wed Aug 17 23:31:53 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:54.155248 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c1b03-1fee-4473-a0d8-98a3d741c3ed None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:54.224497 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:54.257156 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c1b03-1fee-4473-a0d8-98a3d741c3ed None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:54.290879 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c9181-c260-4901-aa99-a5be012f8c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:54.291474 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c9181-c260-4901-aa99-a5be012f8c44 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:54.291749 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c9181-c260-4901-aa99-a5be012f8c44 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:54.292039 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c9181-c260-4901-aa99-a5be012f8c44 None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:54.292655 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c9181-c260-4901-aa99-a5be012f8c44 None admin] RBAC: Adding query filter params (name=image) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:54.292952 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c9181-c260-4901-aa99-a5be012f8c44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:54.294313 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c9181-c260-4901-aa99-a5be012f8c44 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:54.299203 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 492/993] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:54 2022] GET /identity/v3/services?name=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:54.304637 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e10446-f681-48fa-aebb-3da9402aabdd None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:54.405201 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e10446-f681-48fa-aebb-3da9402aabdd None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:54.442309 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2dbe31-b634-4eb7-8cdf-a469f5d7f52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:54.442309 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2dbe31-b634-4eb7-8cdf-a469f5d7f52e None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:54.442309 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2dbe31-b634-4eb7-8cdf-a469f5d7f52e None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:54.442309 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2dbe31-b634-4eb7-8cdf-a469f5d7f52e None admin] PATH_INFO: `/v3/services` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:54.442726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2dbe31-b634-4eb7-8cdf-a469f5d7f52e None admin] RBAC: Adding query filter params (type=image) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:54.442726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2dbe31-b634-4eb7-8cdf-a469f5d7f52e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:54.442870 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2dbe31-b634-4eb7-8cdf-a469f5d7f52e None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:54.460240 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 502/994] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:54 2022] GET /identity/v3/services?type=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:54.572472 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c65487-a875-4b60-8553-1cf481a8623a None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:54.572656 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c65487-a875-4b60-8553-1cf481a8623a None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:54.572931 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c65487-a875-4b60-8553-1cf481a8623a None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:54.574306 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 493/995] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:54.579817 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce5fd1-3cd4-48a8-b31a-f183470e181b None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:54.580141 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce5fd1-3cd4-48a8-b31a-f183470e181b None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:54.580420 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce5fd1-3cd4-48a8-b31a-f183470e181b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:54.598921 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-f3ce5fd1-3cd4-48a8-b31a-f183470e181b None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:54.600397 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-f3ce5fd1-3cd4-48a8-b31a-f183470e181b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:54.647868 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:54.674460 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 503/996] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:54 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:31:54.937887 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c65487-a875-4b60-8553-1cf481a8623a None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:55.044324 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c65487-a875-4b60-8553-1cf481a8623a None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:55.079824 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089129f-8bea-41c4-a904-f4f6f4afa99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:55.080487 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089129f-8bea-41c4-a904-f4f6f4afa99d None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:55.080770 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089129f-8bea-41c4-a904-f4f6f4afa99d None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:55.081044 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089129f-8bea-41c4-a904-f4f6f4afa99d None admin] PATH_INFO: `/v3/services/compute` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:55.081639 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5089129f-8bea-41c4-a904-f4f6f4afa99d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:55.083041 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5089129f-8bea-41c4-a904-f4f6f4afa99d None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:55.085019 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:55.093541 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-5089129f-8bea-41c4-a904-f4f6f4afa99d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 17 23:31:55.093541 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:55.093541 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 23:31:55.093541 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 23:31:55.093541 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 23:31:55.093541 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 23:31:55.093541 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 23:31:55.093541 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:55.093541 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 17 23:31:55.093541 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:55.093541 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 23:31:55.093541 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:55.093541 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:55.093541 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:55.093541 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:55.093541 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:55.093541 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:55.094617 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:55.094617 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:55.094617 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:55.094617 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:55.094617 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:55.094617 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:55.094617 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 23:31:55.094617 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 23:31:55.094617 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 23:31:55.094617 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 23:31:55.094617 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:55.094617 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:55.094617 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:55.094617 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:55.094617 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:55.094617 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:55.096900 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:55.096900 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:55.096900 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:55.096900 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:55.096900 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:55.096900 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:55.096900 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:55.096900 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:55.096900 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:55.096900 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:55.096900 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 23:31:55.096900 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:55.096900 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 17 23:31:55.096900 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:55.096900 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 494/997] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:31:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:55.100529 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce5fd1-3cd4-48a8-b31a-f183470e181b None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:55.197706 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce5fd1-3cd4-48a8-b31a-f183470e181b None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:55.231173 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ef91f8-2e0f-4d0b-bda2-6c789a435937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:55.231781 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef91f8-2e0f-4d0b-bda2-6c789a435937 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:55.232084 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef91f8-2e0f-4d0b-bda2-6c789a435937 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:55.232359 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef91f8-2e0f-4d0b-bda2-6c789a435937 None admin] PATH_INFO: `/v3/services` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:55.233010 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ef91f8-2e0f-4d0b-bda2-6c789a435937 None admin] RBAC: Adding query filter params (name=compute) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:55.233312 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ef91f8-2e0f-4d0b-bda2-6c789a435937 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:55.234684 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ef91f8-2e0f-4d0b-bda2-6c789a435937 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:55.244139 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 504/998] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Aug 17 23:31:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:55.245383 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089129f-8bea-41c4-a904-f4f6f4afa99d None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:55.348102 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089129f-8bea-41c4-a904-f4f6f4afa99d None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:55.381792 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5646db-0494-44cd-8bb2-808dcf2709c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:55.382393 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5646db-0494-44cd-8bb2-808dcf2709c5 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:55.382669 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5646db-0494-44cd-8bb2-808dcf2709c5 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:55.382934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5646db-0494-44cd-8bb2-808dcf2709c5 None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:55.383558 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5646db-0494-44cd-8bb2-808dcf2709c5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:55.383856 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5646db-0494-44cd-8bb2-808dcf2709c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:55.385784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5646db-0494-44cd-8bb2-808dcf2709c5 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:55.392252 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 495/999] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Aug 17 23:31:55 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:55.398999 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ef91f8-2e0f-4d0b-bda2-6c789a435937 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:55.446290 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:55.460729 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8350f1b-e82c-4188-b298-fd87db20a557 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:55.461013 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8350f1b-e82c-4188-b298-fd87db20a557 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:55.461292 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8350f1b-e82c-4188-b298-fd87db20a557 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:55.462960 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 496/1000] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 23:31:55.467596 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ca9b0-8d64-478e-8894-44d687fb502b None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:55.467875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ca9b0-8d64-478e-8894-44d687fb502b None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:55.468184 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ca9b0-8d64-478e-8894-44d687fb502b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:55.486210 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-9d4ca9b0-8d64-478e-8894-44d687fb502b None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:55.487395 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-9d4ca9b0-8d64-478e-8894-44d687fb502b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:55.511547 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ef91f8-2e0f-4d0b-bda2-6c789a435937 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:55.548190 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c9ab6-a3ae-4dfc-9cc9-e8eb9fc579c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:55.548958 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c9ab6-a3ae-4dfc-9cc9-e8eb9fc579c5 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:55.549245 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c9ab6-a3ae-4dfc-9cc9-e8eb9fc579c5 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:55.549678 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:55.549854 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c9ab6-a3ae-4dfc-9cc9-e8eb9fc579c5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:55.550808 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c9ab6-a3ae-4dfc-9cc9-e8eb9fc579c5 None admin] RBAC: Adding query filter params (interface=public, service_id=776a78bba1ba450a855a737880a7a75e, region_id=RegionOne) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:55.551219 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c9ab6-a3ae-4dfc-9cc9-e8eb9fc579c5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:55.553225 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c9ab6-a3ae-4dfc-9cc9-e8eb9fc579c5 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:55.559686 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 505/1001] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Aug 17 23:31:55 2022] GET /identity/v3/endpoints?service_id=776a78bba1ba450a855a737880a7a75e&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:55.562165 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 497/1002] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:55 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 23:31:55.568813 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c9ab6-a3ae-4dfc-9cc9-e8eb9fc579c5 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:55.666717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c9ab6-a3ae-4dfc-9cc9-e8eb9fc579c5 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:55.668965 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:55.703745 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b098e1c-a012-4310-a8ad-a97c4e1f2aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:55.704300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b098e1c-a012-4310-a8ad-a97c4e1f2aee None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:55.704575 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b098e1c-a012-4310-a8ad-a97c4e1f2aee None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:55.704844 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b098e1c-a012-4310-a8ad-a97c4e1f2aee None admin] PATH_INFO: `/v3/services` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:55.705444 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b098e1c-a012-4310-a8ad-a97c4e1f2aee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:55.706789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b098e1c-a012-4310-a8ad-a97c4e1f2aee None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:55.713998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 506/1003] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Aug 17 23:31:55 2022] GET /identity/v3/services => generated 1727 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:31:55.835998 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ca9b0-8d64-478e-8894-44d687fb502b None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:55.936778 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ca9b0-8d64-478e-8894-44d687fb502b None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:55.970380 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd35fcf-7465-4d43-b06c-baabdc1a7d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:55.971017 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd35fcf-7465-4d43-b06c-baabdc1a7d59 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:55.971313 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd35fcf-7465-4d43-b06c-baabdc1a7d59 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:55.971567 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd35fcf-7465-4d43-b06c-baabdc1a7d59 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:55.972175 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd35fcf-7465-4d43-b06c-baabdc1a7d59 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:55.973549 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd35fcf-7465-4d43-b06c-baabdc1a7d59 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:55.982774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-efd35fcf-7465-4d43-b06c-baabdc1a7d59 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 23:31:55.982774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:55.982774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 23:31:55.982774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 23:31:55.982774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 23:31:55.982774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 23:31:55.982774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 23:31:55.982774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:55.982774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 23:31:55.982774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:55.982774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 23:31:55.982774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:55.982774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:55.982774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:55.982774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:55.982774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:55.982774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:55.983747 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:55.983747 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:55.983747 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:55.983747 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:55.983747 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:55.983747 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:55.983747 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 23:31:55.983747 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 23:31:55.983747 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 23:31:55.983747 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 23:31:55.983747 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:55.983747 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:55.983747 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:55.983747 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:55.983747 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:55.983747 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:55.984737 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:55.984737 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:55.984737 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:55.984737 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:55.984737 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:55.984737 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:55.984737 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:55.984737 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:55.984737 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:55.984737 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:55.984737 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 23:31:55.984737 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:55.984737 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 23:31:55.984737 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:31:55.988032 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 498/1004] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Aug 17 23:31:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:55.990117 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b098e1c-a012-4310-a8ad-a97c4e1f2aee None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:56.092806 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b098e1c-a012-4310-a8ad-a97c4e1f2aee None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:56.126992 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6219cd-e753-449a-810c-b2e34b936769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:56.127597 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6219cd-e753-449a-810c-b2e34b936769 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:56.127872 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6219cd-e753-449a-810c-b2e34b936769 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:56.128167 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6219cd-e753-449a-810c-b2e34b936769 None admin] PATH_INFO: `/v3/services` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:56.129411 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6219cd-e753-449a-810c-b2e34b936769 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:56.129741 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6219cd-e753-449a-810c-b2e34b936769 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:56.131086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6219cd-e753-449a-810c-b2e34b936769 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:56.141441 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 507/1005] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Aug 17 23:31:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:56.142994 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd35fcf-7465-4d43-b06c-baabdc1a7d59 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:56.203221 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:56.249182 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd35fcf-7465-4d43-b06c-baabdc1a7d59 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:56.283217 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7d4bc5-dbe3-427f-9f5f-fed4b741afb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:56.283815 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d4bc5-dbe3-427f-9f5f-fed4b741afb8 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:56.284119 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d4bc5-dbe3-427f-9f5f-fed4b741afb8 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:56.284389 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d4bc5-dbe3-427f-9f5f-fed4b741afb8 None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:56.285008 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7d4bc5-dbe3-427f-9f5f-fed4b741afb8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:56.285306 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7d4bc5-dbe3-427f-9f5f-fed4b741afb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:56.286668 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7d4bc5-dbe3-427f-9f5f-fed4b741afb8 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:56.291586 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 499/1006] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Aug 17 23:31:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:31:56.297040 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6219cd-e753-449a-810c-b2e34b936769 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:56.328814 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:56.352667 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73a39b-498c-49fe-89b6-fe629281c498 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:56.352947 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73a39b-498c-49fe-89b6-fe629281c498 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:56.353229 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73a39b-498c-49fe-89b6-fe629281c498 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:56.354701 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 500/1007] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:56.359337 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e53615-59e2-4cf6-abdd-7ec553d0eb4b None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:56.359624 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e53615-59e2-4cf6-abdd-7ec553d0eb4b None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:56.359901 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e53615-59e2-4cf6-abdd-7ec553d0eb4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:56.377833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-55e53615-59e2-4cf6-abdd-7ec553d0eb4b None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:56.379022 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-55e53615-59e2-4cf6-abdd-7ec553d0eb4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:56.399860 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6219cd-e753-449a-810c-b2e34b936769 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:56.434564 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8425e-2cca-4ad1-b52f-2f09e5978416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:56.435272 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8425e-2cca-4ad1-b52f-2f09e5978416 None admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:56.435548 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8425e-2cca-4ad1-b52f-2f09e5978416 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:56.435861 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8425e-2cca-4ad1-b52f-2f09e5978416 None admin] PATH_INFO: `/v3/endpoints` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:56.436553 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8425e-2cca-4ad1-b52f-2f09e5978416 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:56.437919 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8425e-2cca-4ad1-b52f-2f09e5978416 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:56.452013 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 501/1008] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:56 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 23:31:56.459168 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 508/1009] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:31:56 2022] POST /identity/v3/endpoints => generated 325 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:31:56.725705 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e53615-59e2-4cf6-abdd-7ec553d0eb4b None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:56.778484 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:56.832206 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e53615-59e2-4cf6-abdd-7ec553d0eb4b None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:56.872377 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9857c933-7f2b-46dc-b2f0-a6d2e61d5133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:56.872989 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857c933-7f2b-46dc-b2f0-a6d2e61d5133 None admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:56.873282 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857c933-7f2b-46dc-b2f0-a6d2e61d5133 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:56.873541 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857c933-7f2b-46dc-b2f0-a6d2e61d5133 None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:56.874180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9857c933-7f2b-46dc-b2f0-a6d2e61d5133 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:56.875547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9857c933-7f2b-46dc-b2f0-a6d2e61d5133 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:56.892039 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 502/1010] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Aug 17 23:31:56 2022] POST /identity/v3/services => generated 242 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:31:57.612467 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de8d5c-2209-4749-8909-92ae26c57b95 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:57.613024 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de8d5c-2209-4749-8909-92ae26c57b95 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:57.613024 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de8d5c-2209-4749-8909-92ae26c57b95 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:57.614474 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 509/1011] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:31:57.619924 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73d0c3-8733-45f7-9ddf-0633e9f5d121 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:57.620374 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73d0c3-8733-45f7-9ddf-0633e9f5d121 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:57.620534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73d0c3-8733-45f7-9ddf-0633e9f5d121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:57.637879 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-4e73d0c3-8733-45f7-9ddf-0633e9f5d121 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:57.639068 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-4e73d0c3-8733-45f7-9ddf-0633e9f5d121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:57.707679 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 503/1012] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:57 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:31:57.972237 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de8d5c-2209-4749-8909-92ae26c57b95 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:58.068050 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:58.079637 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de8d5c-2209-4749-8909-92ae26c57b95 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:58.112582 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9dd6bf-58cc-4d06-9f90-0b3335f0d60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:58.113209 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9dd6bf-58cc-4d06-9f90-0b3335f0d60f None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:58.113489 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9dd6bf-58cc-4d06-9f90-0b3335f0d60f None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:58.113764 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9dd6bf-58cc-4d06-9f90-0b3335f0d60f None admin] PATH_INFO: `/v3/services/compute` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:58.114343 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9dd6bf-58cc-4d06-9f90-0b3335f0d60f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:58.115714 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9dd6bf-58cc-4d06-9f90-0b3335f0d60f None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:58.124971 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-0a9dd6bf-58cc-4d06-9f90-0b3335f0d60f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 17 23:31:58.124971 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:58.124971 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 23:31:58.124971 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 23:31:58.124971 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 23:31:58.124971 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 23:31:58.124971 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 23:31:58.124971 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:58.124971 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 17 23:31:58.124971 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:58.124971 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 23:31:58.124971 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:58.124971 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:58.124971 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:58.124971 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:58.124971 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:58.124971 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:58.125967 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:58.125967 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:58.125967 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:58.125967 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:58.125967 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:58.125967 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:58.125967 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 23:31:58.125967 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 23:31:58.125967 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 23:31:58.125967 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 23:31:58.125967 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:58.125967 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:58.125967 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:58.125967 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:58.125967 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:58.125967 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:58.126811 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:58.126811 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:58.126811 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:58.126811 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:58.126811 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:58.126811 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:58.126811 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:58.126811 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:58.126811 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:58.126811 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:58.126811 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 23:31:58.126811 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:58.126811 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 17 23:31:58.126811 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:58.126811 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 510/1013] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:31:57 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:58.133503 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73d0c3-8733-45f7-9ddf-0633e9f5d121 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:58.164259 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:58.235867 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73d0c3-8733-45f7-9ddf-0633e9f5d121 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:58.270199 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f68df-e8f8-478f-8d92-ac60ed9ee0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:58.270806 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f68df-e8f8-478f-8d92-ac60ed9ee0aa None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:58.271087 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f68df-e8f8-478f-8d92-ac60ed9ee0aa None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:58.271359 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f68df-e8f8-478f-8d92-ac60ed9ee0aa None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:58.272013 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f68df-e8f8-478f-8d92-ac60ed9ee0aa None admin] RBAC: Adding query filter params (name=compute) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:58.272324 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f68df-e8f8-478f-8d92-ac60ed9ee0aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:58.273696 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f68df-e8f8-478f-8d92-ac60ed9ee0aa None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:58.283595 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 504/1014] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Aug 17 23:31:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:31:58.284795 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9dd6bf-58cc-4d06-9f90-0b3335f0d60f None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:58.364973 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1851520-7173-4e59-97e5-8aa8e592857f None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:58.365318 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1851520-7173-4e59-97e5-8aa8e592857f None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:58.365527 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1851520-7173-4e59-97e5-8aa8e592857f None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:58.366872 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 505/1015] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 23:31:58.372356 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666bf2f-489e-4c8f-89da-07a2d7f2025e None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:58.372639 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666bf2f-489e-4c8f-89da-07a2d7f2025e None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:58.372912 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666bf2f-489e-4c8f-89da-07a2d7f2025e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:58.388778 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9dd6bf-58cc-4d06-9f90-0b3335f0d60f None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:58.390492 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-f666bf2f-489e-4c8f-89da-07a2d7f2025e None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:58.391669 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-f666bf2f-489e-4c8f-89da-07a2d7f2025e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:58.424257 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2ed23d-62fe-4cfd-9d86-40c0000ca413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:58.424864 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ed23d-62fe-4cfd-9d86-40c0000ca413 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:58.425403 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ed23d-62fe-4cfd-9d86-40c0000ca413 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:58.425676 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ed23d-62fe-4cfd-9d86-40c0000ca413 None admin] PATH_INFO: `/v3/services` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:58.426298 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2ed23d-62fe-4cfd-9d86-40c0000ca413 None admin] RBAC: Adding query filter params (type=compute) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:58.426597 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2ed23d-62fe-4cfd-9d86-40c0000ca413 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:58.427948 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2ed23d-62fe-4cfd-9d86-40c0000ca413 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:58.432982 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 511/1016] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Aug 17 23:31:58 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:58.438312 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2ed23d-62fe-4cfd-9d86-40c0000ca413 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:58.466385 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:58.468462 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 506/1017] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:58 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 23:31:58.544698 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2ed23d-62fe-4cfd-9d86-40c0000ca413 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:58.577469 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af2af88-3b3b-4ebf-9612-b743c5f0d879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:58.578071 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af2af88-3b3b-4ebf-9612-b743c5f0d879 None admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:58.578348 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af2af88-3b3b-4ebf-9612-b743c5f0d879 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:58.578614 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af2af88-3b3b-4ebf-9612-b743c5f0d879 None admin] PATH_INFO: `/v3/endpoints` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:58.579259 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af2af88-3b3b-4ebf-9612-b743c5f0d879 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:58.580639 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af2af88-3b3b-4ebf-9612-b743c5f0d879 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:58.607990 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 512/1018] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:31:58 2022] POST /identity/v3/endpoints => generated 334 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:31:58.738960 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f666bf2f-489e-4c8f-89da-07a2d7f2025e None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:58.767613 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:58.806441 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa0cca-da61-4609-8b0d-c995d8061fb2 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:58.806923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa0cca-da61-4609-8b0d-c995d8061fb2 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:58.806923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa0cca-da61-4609-8b0d-c995d8061fb2 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:58.808310 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 513/1019] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:31:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 23:31:58.815628 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be75728-50c1-4e8d-bf03-824178c5ed71 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:58.816021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be75728-50c1-4e8d-bf03-824178c5ed71 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:58.816217 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be75728-50c1-4e8d-bf03-824178c5ed71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:58.836142 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-0be75728-50c1-4e8d-bf03-824178c5ed71 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:31:58.837908 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-0be75728-50c1-4e8d-bf03-824178c5ed71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:31:58.849480 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f666bf2f-489e-4c8f-89da-07a2d7f2025e None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:58.897490 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3912f1-1877-4ee6-9a43-1f0add342de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:58.898145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3912f1-1877-4ee6-9a43-1f0add342de2 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:58.898750 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3912f1-1877-4ee6-9a43-1f0add342de2 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:58.899029 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3912f1-1877-4ee6-9a43-1f0add342de2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:58.900211 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3912f1-1877-4ee6-9a43-1f0add342de2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:58.901667 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3912f1-1877-4ee6-9a43-1f0add342de2 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:58.903505 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 507/1020] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:58.909634 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3912f1-1877-4ee6-9a43-1f0add342de2 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:58.924297 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 514/1021] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:31:58 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 23:31:59.007293 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:59.018192 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3912f1-1877-4ee6-9a43-1f0add342de2 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:59.053005 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd08e8a-52b1-4d9d-bd39-f7e4b50acdbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:59.053614 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd08e8a-52b1-4d9d-bd39-f7e4b50acdbd None admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:59.053894 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd08e8a-52b1-4d9d-bd39-f7e4b50acdbd None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:59.054166 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd08e8a-52b1-4d9d-bd39-f7e4b50acdbd None admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:59.054809 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd08e8a-52b1-4d9d-bd39-f7e4b50acdbd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:59.056900 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd08e8a-52b1-4d9d-bd39-f7e4b50acdbd None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:59.091996 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 508/1022] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Aug 17 23:31:58 2022] POST /identity/v3/projects => generated 311 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:31:59.191988 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be75728-50c1-4e8d-bf03-824178c5ed71 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:59.200628 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:59.304593 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be75728-50c1-4e8d-bf03-824178c5ed71 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:59.339252 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fabee5-3b64-464f-999c-e7e7d2b5f72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:59.339804 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fabee5-3b64-464f-999c-e7e7d2b5f72f None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:59.340114 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fabee5-3b64-464f-999c-e7e7d2b5f72f None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:59.340389 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fabee5-3b64-464f-999c-e7e7d2b5f72f None admin] PATH_INFO: `/v3/services/image` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:59.340974 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fabee5-3b64-464f-999c-e7e7d2b5f72f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:59.342360 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fabee5-3b64-464f-999c-e7e7d2b5f72f None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:59.351070 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-40fabee5-3b64-464f-999c-e7e7d2b5f72f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 17 23:31:59.351070 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:59.351070 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 23:31:59.351070 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 23:31:59.351070 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 23:31:59.351070 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 23:31:59.351070 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 23:31:59.351070 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:59.351070 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 17 23:31:59.351070 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:59.351070 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 23:31:59.351070 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:59.351070 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:31:59.351070 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:31:59.351070 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:31:59.351070 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:31:59.351070 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:31:59.352074 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:31:59.352074 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:31:59.352074 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:31:59.352074 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:31:59.352074 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:31:59.352074 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:31:59.352074 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 23:31:59.352074 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 23:31:59.352074 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 23:31:59.352074 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 23:31:59.352074 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:31:59.352074 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:31:59.352074 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:31:59.352074 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:31:59.352074 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:31:59.352074 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:31:59.353572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:31:59.353572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:31:59.353572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:31:59.353572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:31:59.353572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:31:59.353572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:31:59.353572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:31:59.353572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:31:59.353572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:31:59.353572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:31:59.353572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 23:31:59.353572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:31:59.353572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 17 23:31:59.353572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:31:59.353572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 515/1023] 10.222.0.26 () {56 vars in 1105 bytes} [Wed Aug 17 23:31:59 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:31:59.360294 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd08e8a-52b1-4d9d-bd39-f7e4b50acdbd None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:59.464208 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd08e8a-52b1-4d9d-bd39-f7e4b50acdbd None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:59.497983 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be97ec2-0eed-4016-9c7a-890ac1be9f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:59.498586 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be97ec2-0eed-4016-9c7a-890ac1be9f75 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:59.498865 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be97ec2-0eed-4016-9c7a-890ac1be9f75 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:59.499132 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be97ec2-0eed-4016-9c7a-890ac1be9f75 None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:59.499760 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be97ec2-0eed-4016-9c7a-890ac1be9f75 None admin] RBAC: Adding query filter params (name=image) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:59.500087 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be97ec2-0eed-4016-9c7a-890ac1be9f75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:59.501461 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be97ec2-0eed-4016-9c7a-890ac1be9f75 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:59.506430 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 509/1024] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:31:59.512483 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fabee5-3b64-464f-999c-e7e7d2b5f72f None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:59.584291 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:59.617395 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fabee5-3b64-464f-999c-e7e7d2b5f72f None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:59.651106 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a0d6ec-8f00-4871-ae02-38c9f4de1c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:59.651716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0d6ec-8f00-4871-ae02-38c9f4de1c00 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:59.652022 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0d6ec-8f00-4871-ae02-38c9f4de1c00 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:59.652300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0d6ec-8f00-4871-ae02-38c9f4de1c00 None admin] PATH_INFO: `/v3/services` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:59.653178 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a0d6ec-8f00-4871-ae02-38c9f4de1c00 None admin] RBAC: Adding query filter params (type=image) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:59.653480 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a0d6ec-8f00-4871-ae02-38c9f4de1c00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:59.654848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a0d6ec-8f00-4871-ae02-38c9f4de1c00 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:59.664882 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 516/1025] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:31:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:31:59.665720 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be97ec2-0eed-4016-9c7a-890ac1be9f75 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:59.675039 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:59.770254 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be97ec2-0eed-4016-9c7a-890ac1be9f75 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:59.806401 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef55521f-545c-4c14-a9ee-e6f9aefcbd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:59.807008 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55521f-545c-4c14-a9ee-e6f9aefcbd45 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:59.807289 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55521f-545c-4c14-a9ee-e6f9aefcbd45 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:59.807561 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55521f-545c-4c14-a9ee-e6f9aefcbd45 None admin] PATH_INFO: `/v3/endpoints` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:59.808297 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef55521f-545c-4c14-a9ee-e6f9aefcbd45 None admin] RBAC: Adding query filter params (interface=public, service_id=fdd52ab221234900809e0cfb97a978c3, region_id=RegionOne) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:31:59.808607 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef55521f-545c-4c14-a9ee-e6f9aefcbd45 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:59.809974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef55521f-545c-4c14-a9ee-e6f9aefcbd45 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:59.819677 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 510/1026] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Aug 17 23:31:59 2022] GET /identity/v3/endpoints?service_id=fdd52ab221234900809e0cfb97a978c3&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:31:59.821317 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a0d6ec-8f00-4871-ae02-38c9f4de1c00 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:31:59.920121 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a0d6ec-8f00-4871-ae02-38c9f4de1c00 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:31:59.952226 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52501af4-cceb-4605-9d74-c23ca03ad2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:31:59.952829 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52501af4-cceb-4605-9d74-c23ca03ad2a9 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:31:59.953109 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52501af4-cceb-4605-9d74-c23ca03ad2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:31:59.953379 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52501af4-cceb-4605-9d74-c23ca03ad2a9 None admin] PATH_INFO: `/v3/services` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:31:59.953983 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52501af4-cceb-4605-9d74-c23ca03ad2a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:31:59.955337 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52501af4-cceb-4605-9d74-c23ca03ad2a9 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:31:59.956386 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:31:59.963841 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 517/1027] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Aug 17 23:31:59 2022] GET /identity/v3/services => generated 1957 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 23:32:00.628636 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591f8c1-b981-4ab9-a998-f1c4d6549a8e None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:00.628972 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591f8c1-b981-4ab9-a998-f1c4d6549a8e None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:00.629201 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591f8c1-b981-4ab9-a998-f1c4d6549a8e None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:00.630690 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 511/1028] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:32:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:32:00.636196 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cbccc-c5b7-4251-bfcf-3c9793eda8c5 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:00.636490 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cbccc-c5b7-4251-bfcf-3c9793eda8c5 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:00.636770 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cbccc-c5b7-4251-bfcf-3c9793eda8c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:00.654667 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-cc7cbccc-c5b7-4251-bfcf-3c9793eda8c5 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:32:00.655853 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-cc7cbccc-c5b7-4251-bfcf-3c9793eda8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:32:00.721109 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 518/1029] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:32:00 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:32:00.998540 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5591f8c1-b981-4ab9-a998-f1c4d6549a8e None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:01.046052 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2485c-c2d3-4492-a3a7-da55cf8376a2 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:01.046566 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2485c-c2d3-4492-a3a7-da55cf8376a2 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:01.046677 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2485c-c2d3-4492-a3a7-da55cf8376a2 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:01.047934 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 519/1030] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:32:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 23:32:01.052690 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2eab2-84d1-4f8a-8472-dc27cc673aa5 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:01.052985 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2eab2-84d1-4f8a-8472-dc27cc673aa5 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:01.053247 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2eab2-84d1-4f8a-8472-dc27cc673aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:01.072184 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-59a2eab2-84d1-4f8a-8472-dc27cc673aa5 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:32:01.073381 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-59a2eab2-84d1-4f8a-8472-dc27cc673aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:32:01.075308 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:01.109265 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5591f8c1-b981-4ab9-a998-f1c4d6549a8e None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:01.141893 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 520/1031] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:32:01 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 23:32:01.146432 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a938a3cd-96de-4bc2-a7c8-77c3470523b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:01.147053 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938a3cd-96de-4bc2-a7c8-77c3470523b5 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:01.147330 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938a3cd-96de-4bc2-a7c8-77c3470523b5 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:01.147598 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938a3cd-96de-4bc2-a7c8-77c3470523b5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:01.158172 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a938a3cd-96de-4bc2-a7c8-77c3470523b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:32:01.162835 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-a938a3cd-96de-4bc2-a7c8-77c3470523b5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 17 23:32:01.162835 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:32:01.162835 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:32:01.162835 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:32:01.162835 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:32:01.162835 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:32:01.162835 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:32:01.162835 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:32:01.162835 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:32:01.162835 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:32:01.162835 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:32:01.162835 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:32:01.162835 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:32:01.162835 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:32:01.162835 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:32:01.162835 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:32:01.163787 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:32:01.163787 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:32:01.163787 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:32:01.163787 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:32:01.163787 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:32:01.163787 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:32:01.163787 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:32:01.163787 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:32:01.163787 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:32:01.163787 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:32:01.163787 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:32:01.163787 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:32:01.163787 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:32:01.163787 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:32:01.163787 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:32:01.163787 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:32:01.168981 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:32:01.168981 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:32:01.168981 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:32:01.168981 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:32:01.168981 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:32:01.168981 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:32:01.168981 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:32:01.168981 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:32:01.168981 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:32:01.168981 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:32:01.168981 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 17 23:32:01.168981 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:32:01.168981 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 512/1032] 10.222.0.26 () {56 vars in 1120 bytes} [Wed Aug 17 23:32:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:32:01.170957 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2eab2-84d1-4f8a-8472-dc27cc673aa5 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:01.230472 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:01.273071 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2eab2-84d1-4f8a-8472-dc27cc673aa5 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:01.307732 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32bc33-a69d-4cc0-9674-131f8f6b9873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:01.308370 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32bc33-a69d-4cc0-9674-131f8f6b9873 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:01.308652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32bc33-a69d-4cc0-9674-131f8f6b9873 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:01.308927 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32bc33-a69d-4cc0-9674-131f8f6b9873 None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:01.309599 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32bc33-a69d-4cc0-9674-131f8f6b9873 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:32:01.309903 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32bc33-a69d-4cc0-9674-131f8f6b9873 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:01.311272 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32bc33-a69d-4cc0-9674-131f8f6b9873 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:01.319346 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 521/1033] 10.222.0.26 () {56 vars in 1115 bytes} [Wed Aug 17 23:32:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:01.324933 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a938a3cd-96de-4bc2-a7c8-77c3470523b5 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:01.407323 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32bc33-a69d-4cc0-9674-131f8f6b9873 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:01.426745 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a938a3cd-96de-4bc2-a7c8-77c3470523b5 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:01.468000 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:01.475222 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc67875-5ea7-468b-a68b-500b9cd4d581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:01.475886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc67875-5ea7-468b-a68b-500b9cd4d581 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:01.476199 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc67875-5ea7-468b-a68b-500b9cd4d581 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:01.476475 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc67875-5ea7-468b-a68b-500b9cd4d581 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:01.487195 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc67875-5ea7-468b-a68b-500b9cd4d581 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:01.488671 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc67875-5ea7-468b-a68b-500b9cd4d581 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:01.503833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-7bc67875-5ea7-468b-a68b-500b9cd4d581 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:32:01.503833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:32:01.503833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:32:01.503833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:32:01.503833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:32:01.503833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:32:01.503833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:32:01.503833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:32:01.503833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:32:01.503833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:32:01.503833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:32:01.503833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:32:01.503833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:32:01.503833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:32:01.503833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:32:01.503833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:32:01.504944 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:32:01.504944 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:32:01.504944 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:32:01.504944 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:32:01.504944 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:32:01.504944 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:32:01.504944 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:32:01.504944 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:32:01.504944 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:32:01.504944 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:32:01.504944 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:32:01.504944 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:32:01.504944 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:32:01.504944 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:32:01.504944 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:32:01.504944 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:32:01.504944 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:32:01.505858 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:32:01.505858 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:32:01.505858 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:32:01.505858 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 513/1034] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:32:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:32:01.510784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc67875-5ea7-468b-a68b-500b9cd4d581 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:01.518649 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32bc33-a69d-4cc0-9674-131f8f6b9873 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:01.539857 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:01.556468 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2568079f-a5c0-496c-9f44-4dd74ed8c0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:01.557101 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568079f-a5c0-496c-9f44-4dd74ed8c0f2 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:01.557378 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568079f-a5c0-496c-9f44-4dd74ed8c0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:01.557647 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568079f-a5c0-496c-9f44-4dd74ed8c0f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:01.558750 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2568079f-a5c0-496c-9f44-4dd74ed8c0f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:01.560228 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2568079f-a5c0-496c-9f44-4dd74ed8c0f2 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:01.561957 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 522/1035] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:32:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:32:01.567279 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2568079f-a5c0-496c-9f44-4dd74ed8c0f2 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:01.621655 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc67875-5ea7-468b-a68b-500b9cd4d581 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:01.657781 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04272867-801a-43c3-a62c-49f52792b440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:01.658389 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04272867-801a-43c3-a62c-49f52792b440 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:01.658668 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04272867-801a-43c3-a62c-49f52792b440 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:01.658936 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04272867-801a-43c3-a62c-49f52792b440 None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:01.659581 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04272867-801a-43c3-a62c-49f52792b440 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:32:01.660052 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04272867-801a-43c3-a62c-49f52792b440 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:01.661430 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04272867-801a-43c3-a62c-49f52792b440 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:01.678745 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 514/1036] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:32:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:32:01.680234 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04272867-801a-43c3-a62c-49f52792b440 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:01.683320 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2568079f-a5c0-496c-9f44-4dd74ed8c0f2 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:01.720100 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316a8607-46d0-4406-955a-4a00365cc2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:01.720714 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a8607-46d0-4406-955a-4a00365cc2b2 None admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:01.720996 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a8607-46d0-4406-955a-4a00365cc2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:01.721263 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a8607-46d0-4406-955a-4a00365cc2b2 None admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:01.721909 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316a8607-46d0-4406-955a-4a00365cc2b2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:01.723339 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316a8607-46d0-4406-955a-4a00365cc2b2 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:01.727361 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:01.748400 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 523/1037] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Aug 17 23:32:01 2022] POST /identity/v3/users => generated 285 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:32:01.794500 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04272867-801a-43c3-a62c-49f52792b440 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:01.832271 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f07004-9eb4-4fd1-b2c8-041ecd28235f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:01.832991 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f07004-9eb4-4fd1-b2c8-041ecd28235f None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:01.833193 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f07004-9eb4-4fd1-b2c8-041ecd28235f None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:01.833542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f07004-9eb4-4fd1-b2c8-041ecd28235f None admin] PATH_INFO: `/v3/users/nova` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:01.842600 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f07004-9eb4-4fd1-b2c8-041ecd28235f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:01.844099 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f07004-9eb4-4fd1-b2c8-041ecd28235f None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:01.857329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-38f07004-9eb4-4fd1-b2c8-041ecd28235f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 17 23:32:01.857329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:32:01.857329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:32:01.857329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:32:01.857329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:32:01.857329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:32:01.857329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:32:01.857329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:32:01.857329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:32:01.857329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:32:01.857329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:32:01.857329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:32:01.857329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 23:32:01.857329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 23:32:01.857329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 23:32:01.857329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 23:32:01.858428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:32:01.858428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:32:01.858428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 23:32:01.858428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:32:01.858428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 23:32:01.858428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:32:01.858428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:32:01.858428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:32:01.858428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:32:01.858428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:32:01.858428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:32:01.858428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:32:01.858428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:32:01.858428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:32:01.858428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:32:01.858428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:32:01.858428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:32:01.858428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:32:01.859408 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:32:01.859408 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:32:01.859408 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 23:32:01.859408 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 23:32:01.859408 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 23:32:01.859408 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 23:32:01.859408 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 23:32:01.859408 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 23:32:01.859408 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 17 23:32:01.859408 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:32:01.859408 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 515/1038] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Aug 17 23:32:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:32:01.864009 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8687c-ef3e-4c36-8b53-ae895e4bc757 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:01.864009 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8687c-ef3e-4c36-8b53-ae895e4bc757 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:01.864009 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8687c-ef3e-4c36-8b53-ae895e4bc757 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:01.864286 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f07004-9eb4-4fd1-b2c8-041ecd28235f None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:01.865139 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 524/1039] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:32:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:32:01.871000 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff9780-7ca2-4bc3-aa8e-1c5cf8488f00 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:01.871282 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff9780-7ca2-4bc3-aa8e-1c5cf8488f00 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:01.871549 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff9780-7ca2-4bc3-aa8e-1c5cf8488f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:01.890372 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-b4ff9780-7ca2-4bc3-aa8e-1c5cf8488f00 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:32:01.891582 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-b4ff9780-7ca2-4bc3-aa8e-1c5cf8488f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:32:01.938606 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:01.971265 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 525/1040] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:32:01 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 23:32:01.979596 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f07004-9eb4-4fd1-b2c8-041ecd28235f None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:01.981373 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:02.017897 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d12404-052c-45e4-a1f5-221cc3c34559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:02.018504 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d12404-052c-45e4-a1f5-221cc3c34559 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:02.018767 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d12404-052c-45e4-a1f5-221cc3c34559 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:02.019086 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d12404-052c-45e4-a1f5-221cc3c34559 None admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:02.019821 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d12404-052c-45e4-a1f5-221cc3c34559 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:32:02.020207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d12404-052c-45e4-a1f5-221cc3c34559 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:02.021649 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d12404-052c-45e4-a1f5-221cc3c34559 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:02.040467 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 516/1041] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Aug 17 23:32:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:32:02.046079 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff9780-7ca2-4bc3-aa8e-1c5cf8488f00 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:02.147879 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff9780-7ca2-4bc3-aa8e-1c5cf8488f00 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:02.182206 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f30d2fa-9686-4e1f-a41f-ab1162efd1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:02.182821 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30d2fa-9686-4e1f-a41f-ab1162efd1c8 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:02.183103 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30d2fa-9686-4e1f-a41f-ab1162efd1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:02.183375 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30d2fa-9686-4e1f-a41f-ab1162efd1c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:02.225594 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:02.241774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d12404-052c-45e4-a1f5-221cc3c34559 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:02.254082 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f30d2fa-9686-4e1f-a41f-ab1162efd1c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:02.255880 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f30d2fa-9686-4e1f-a41f-ab1162efd1c8 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:02.348544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d12404-052c-45e4-a1f5-221cc3c34559 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:02.360759 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 526/1042] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:32:02 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 23:32:02.366005 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f30d2fa-9686-4e1f-a41f-ab1162efd1c8 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:02.388757 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9824ed20-7547-415e-ac12-366024a6a03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:02.389393 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824ed20-7547-415e-ac12-366024a6a03e None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:02.389680 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824ed20-7547-415e-ac12-366024a6a03e None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:02.389954 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824ed20-7547-415e-ac12-366024a6a03e None admin] PATH_INFO: `/v3/services/image` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:02.390533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9824ed20-7547-415e-ac12-366024a6a03e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:02.391914 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9824ed20-7547-415e-ac12-366024a6a03e None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:02.400770 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-9824ed20-7547-415e-ac12-366024a6a03e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 17 23:32:02.400770 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:32:02.400770 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 23:32:02.400770 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 23:32:02.400770 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 23:32:02.400770 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 23:32:02.400770 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 23:32:02.400770 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:32:02.400770 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 17 23:32:02.400770 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:32:02.400770 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 23:32:02.400770 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:32:02.400770 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:32:02.400770 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:32:02.400770 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:32:02.400770 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:32:02.400770 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:32:02.401784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:32:02.401784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:32:02.401784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:32:02.401784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:32:02.401784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:32:02.401784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:32:02.401784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 23:32:02.401784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 23:32:02.401784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 23:32:02.401784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 23:32:02.401784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:32:02.401784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:32:02.401784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:32:02.401784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:32:02.401784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:32:02.401784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:32:02.402641 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:32:02.402641 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:32:02.402641 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:32:02.402641 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:32:02.402641 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:32:02.402641 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:32:02.402641 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:32:02.402641 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:32:02.402641 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:32:02.402641 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:32:02.402641 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 23:32:02.402641 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:32:02.402641 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 17 23:32:02.402641 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:32:02.402641 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 517/1043] 10.222.0.26 () {56 vars in 1105 bytes} [Wed Aug 17 23:32:02 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:32:02.407834 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9824ed20-7547-415e-ac12-366024a6a03e None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:02.432914 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:02.454198 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:02.480647 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f30d2fa-9686-4e1f-a41f-ab1162efd1c8 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:02.515208 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9824ed20-7547-415e-ac12-366024a6a03e None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:02.518419 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d2d809-957c-46ae-9b60-35b84ac6f1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:02.519052 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2d809-957c-46ae-9b60-35b84ac6f1cb None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:02.519335 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2d809-957c-46ae-9b60-35b84ac6f1cb None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:02.519607 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2d809-957c-46ae-9b60-35b84ac6f1cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:02.530302 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d2d809-957c-46ae-9b60-35b84ac6f1cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:02.531745 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d2d809-957c-46ae-9b60-35b84ac6f1cb None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:02.547137 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-f7d2d809-957c-46ae-9b60-35b84ac6f1cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:32:02.547137 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:32:02.547137 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:32:02.547137 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:32:02.547137 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:32:02.547137 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:32:02.547137 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:32:02.547137 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:32:02.547137 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:32:02.547137 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:32:02.547137 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:32:02.547137 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:32:02.547137 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:32:02.547137 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:32:02.547137 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:32:02.547137 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:32:02.548110 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:32:02.548110 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:32:02.548110 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:32:02.548110 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:32:02.548110 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:32:02.548110 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:32:02.548110 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:32:02.548110 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:32:02.548110 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:32:02.548110 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:32:02.548110 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:32:02.548110 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:32:02.548110 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:32:02.548110 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:32:02.548110 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:32:02.548110 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:32:02.548110 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:32:02.549016 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:32:02.549016 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:32:02.549016 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:32:02.549016 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 527/1044] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:32:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:32:02.553879 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d2d809-957c-46ae-9b60-35b84ac6f1cb None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:02.555069 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7e8ad-459f-4e86-8879-2b28163d7106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:02.556244 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7e8ad-459f-4e86-8879-2b28163d7106 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:02.556528 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7e8ad-459f-4e86-8879-2b28163d7106 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:02.556804 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7e8ad-459f-4e86-8879-2b28163d7106 None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:02.557438 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7e8ad-459f-4e86-8879-2b28163d7106 None admin] RBAC: Adding query filter params (name=image) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:32:02.557742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7e8ad-459f-4e86-8879-2b28163d7106 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:02.559111 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7e8ad-459f-4e86-8879-2b28163d7106 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:02.564450 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 518/1045] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:32:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:32:02.569972 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7e8ad-459f-4e86-8879-2b28163d7106 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:02.660328 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d2d809-957c-46ae-9b60-35b84ac6f1cb None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:02.668398 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:02.681391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7e8ad-459f-4e86-8879-2b28163d7106 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:02.689937 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:02.700197 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd485c19-54c0-4174-b8e1-c512029eee7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:02.700809 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd485c19-54c0-4174-b8e1-c512029eee7b None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:02.701085 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd485c19-54c0-4174-b8e1-c512029eee7b None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:02.701479 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd485c19-54c0-4174-b8e1-c512029eee7b None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:02.702120 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd485c19-54c0-4174-b8e1-c512029eee7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:32:02.702423 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd485c19-54c0-4174-b8e1-c512029eee7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:02.703785 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd485c19-54c0-4174-b8e1-c512029eee7b None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:02.716332 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec24a1a1-4261-47b8-8c03-9dbc74d05c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:02.716935 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24a1a1-4261-47b8-8c03-9dbc74d05c4b None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:02.721601 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24a1a1-4261-47b8-8c03-9dbc74d05c4b None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:02.721702 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 528/1046] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:32:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:32:02.721797 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24a1a1-4261-47b8-8c03-9dbc74d05c4b None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:02.721797 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec24a1a1-4261-47b8-8c03-9dbc74d05c4b None admin] RBAC: Adding query filter params (type=image) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:32:02.721797 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec24a1a1-4261-47b8-8c03-9dbc74d05c4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:02.721797 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec24a1a1-4261-47b8-8c03-9dbc74d05c4b None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:02.722458 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd485c19-54c0-4174-b8e1-c512029eee7b None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:02.726184 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 519/1047] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:32:02 2022] GET /identity/v3/services?type=image => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:32:02.732186 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec24a1a1-4261-47b8-8c03-9dbc74d05c4b None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:02.830445 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd485c19-54c0-4174-b8e1-c512029eee7b None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:02.836255 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec24a1a1-4261-47b8-8c03-9dbc74d05c4b None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:02.871989 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f419f13-97b8-4d58-9990-173119193d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:02.872619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f419f13-97b8-4d58-9990-173119193d2d None admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:02.872900 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f419f13-97b8-4d58-9990-173119193d2d None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:02.873172 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f419f13-97b8-4d58-9990-173119193d2d None admin] PATH_INFO: `/v3/endpoints` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:02.873819 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0350454-665c-4009-b29f-abbf12dd3507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:02.873939 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f419f13-97b8-4d58-9990-173119193d2d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:02.874545 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0350454-665c-4009-b29f-abbf12dd3507 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:02.874898 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0350454-665c-4009-b29f-abbf12dd3507 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:02.875228 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f419f13-97b8-4d58-9990-173119193d2d None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:02.875321 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0350454-665c-4009-b29f-abbf12dd3507 None admin] PATH_INFO: `/v3/projects/service` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:02.883094 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0350454-665c-4009-b29f-abbf12dd3507 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:02.884631 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0350454-665c-4009-b29f-abbf12dd3507 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:02.897707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-c0350454-665c-4009-b29f-abbf12dd3507 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:32:02.897707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:32:02.897707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:32:02.897707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:32:02.897707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:32:02.897707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:32:02.897707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:32:02.897707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:32:02.897707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:32:02.897707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:32:02.897707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:32:02.897707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:32:02.897707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 23:32:02.897707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 23:32:02.897707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 23:32:02.898847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 23:32:02.898847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:32:02.898847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:32:02.898847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:32:02.898847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:32:02.898847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:32:02.898847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:32:02.898847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:32:02.898847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:32:02.898847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:32:02.898847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:32:02.898847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:32:02.898847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:32:02.898847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:32:02.898847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:32:02.898847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:32:02.898847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:32:02.900493 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 23:32:02.900493 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 23:32:02.900493 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 23:32:02.900493 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 23:32:02.900493 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 23:32:02.900493 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 23:32:02.900493 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:32:02.900493 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:32:02.900493 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 529/1048] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:32:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:32:02.912800 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 520/1049] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:32:02 2022] POST /identity/v3/endpoints => generated 327 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:32:02.913445 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0350454-665c-4009-b29f-abbf12dd3507 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:02.969371 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:03.028573 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0350454-665c-4009-b29f-abbf12dd3507 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:03.076666 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8746ea-ffa8-47f4-9156-215516bbdeba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:03.077316 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8746ea-ffa8-47f4-9156-215516bbdeba None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:03.077576 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8746ea-ffa8-47f4-9156-215516bbdeba None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:03.077847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8746ea-ffa8-47f4-9156-215516bbdeba None admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:03.078531 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8746ea-ffa8-47f4-9156-215516bbdeba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:32:03.078832 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8746ea-ffa8-47f4-9156-215516bbdeba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:03.080288 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8746ea-ffa8-47f4-9156-215516bbdeba None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:03.093026 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 530/1050] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 17 23:32:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:32:03.099267 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f419f13-97b8-4d58-9990-173119193d2d None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:03.111432 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:03.204491 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f419f13-97b8-4d58-9990-173119193d2d None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:03.238761 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a750d-d1cb-4c60-86af-ba1c3a1fc25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:03.239369 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a750d-d1cb-4c60-86af-ba1c3a1fc25b None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:03.239649 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a750d-d1cb-4c60-86af-ba1c3a1fc25b None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:03.239919 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a750d-d1cb-4c60-86af-ba1c3a1fc25b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:03.240684 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64a750d-d1cb-4c60-86af-ba1c3a1fc25b None admin] RBAC: Adding query filter params (role.id=dc16b44096d040dfb737c287986be85e, scope.project.id=4de86915ef4340248ba8a8f51d56594e, user.id=ec1dcbe43b0749c3bcc4af9100d771d1) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:32:03.240989 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64a750d-d1cb-4c60-86af-ba1c3a1fc25b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:03.242415 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64a750d-d1cb-4c60-86af-ba1c3a1fc25b None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:03.255990 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 521/1051] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:32:03 2022] GET /identity/v3/role_assignments?user.id=ec1dcbe43b0749c3bcc4af9100d771d1&scope.project.id=4de86915ef4340248ba8a8f51d56594e&role.id=dc16b44096d040dfb737c287986be85e => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:03.716637 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54b39e-4416-4d95-ac0d-72024c1f1d56 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:03.716998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54b39e-4416-4d95-ac0d-72024c1f1d56 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:03.717189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54b39e-4416-4d95-ac0d-72024c1f1d56 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:03.718656 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 531/1052] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:32:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:32:03.723721 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbebdf2-b26c-43af-a023-b45381f851b4 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:03.724031 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbebdf2-b26c-43af-a023-b45381f851b4 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:03.724416 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbebdf2-b26c-43af-a023-b45381f851b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:03.743702 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-afbebdf2-b26c-43af-a023-b45381f851b4 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:32:03.744899 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-afbebdf2-b26c-43af-a023-b45381f851b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:32:03.813592 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 522/1053] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:32:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:32:04.079149 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b54b39e-4416-4d95-ac0d-72024c1f1d56 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:04.180180 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b54b39e-4416-4d95-ac0d-72024c1f1d56 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:04.181819 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:04.215215 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51964994-5f33-47b5-8a57-5c7cb5d72ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:04.215843 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51964994-5f33-47b5-8a57-5c7cb5d72ab0 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:04.216152 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51964994-5f33-47b5-8a57-5c7cb5d72ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:04.216428 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51964994-5f33-47b5-8a57-5c7cb5d72ab0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:04.226945 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51964994-5f33-47b5-8a57-5c7cb5d72ab0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:32:04.231714 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-51964994-5f33-47b5-8a57-5c7cb5d72ab0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:32:04.231714 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:32:04.231714 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:32:04.231714 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:32:04.231714 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:32:04.231714 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:32:04.231714 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:32:04.231714 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:32:04.231714 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:32:04.231714 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:32:04.231714 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:32:04.231714 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:32:04.231714 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:32:04.231714 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:32:04.231714 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:32:04.231714 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:32:04.232665 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:32:04.232665 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:32:04.232665 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:32:04.232665 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:32:04.232665 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:32:04.232665 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:32:04.232665 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:32:04.232665 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:32:04.232665 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:32:04.232665 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:32:04.232665 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:32:04.232665 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:32:04.232665 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:32:04.232665 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:32:04.232665 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:32:04.232665 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:32:04.233537 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:32:04.233537 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:32:04.233537 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:32:04.233537 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:32:04.233537 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:32:04.233537 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:32:04.233537 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:32:04.233537 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:32:04.233537 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:32:04.233537 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:32:04.233537 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:32:04.233537 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:32:04.236361 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 532/1054] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 17 23:32:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:32:04.238869 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbebdf2-b26c-43af-a023-b45381f851b4 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:04.241103 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:04.341477 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbebdf2-b26c-43af-a023-b45381f851b4 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:04.374699 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c55e508-bd05-44ba-b711-91f93facc94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:04.375296 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55e508-bd05-44ba-b711-91f93facc94d None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:04.375571 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55e508-bd05-44ba-b711-91f93facc94d None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:04.375836 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55e508-bd05-44ba-b711-91f93facc94d None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:04.376519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c55e508-bd05-44ba-b711-91f93facc94d None admin] RBAC: Adding query filter params (name=admin) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:32:04.376819 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c55e508-bd05-44ba-b711-91f93facc94d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:04.378207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c55e508-bd05-44ba-b711-91f93facc94d None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:04.391449 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 523/1055] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:32:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:32:04.392824 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51964994-5f33-47b5-8a57-5c7cb5d72ab0 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:04.497457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51964994-5f33-47b5-8a57-5c7cb5d72ab0 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:04.530638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b09a36-418d-427f-a4ab-3afc0bf56117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:04.531261 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b09a36-418d-427f-a4ab-3afc0bf56117 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:04.531537 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b09a36-418d-427f-a4ab-3afc0bf56117 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:04.531806 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b09a36-418d-427f-a4ab-3afc0bf56117 None admin] PATH_INFO: `/v3/users/f94a201f7b784d3e8937d496487e60cb` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:04.545020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b09a36-418d-427f-a4ab-3afc0bf56117 None admin] RBAC: Authorizing `identity:get_user(user_id=f94a201f7b784d3e8937d496487e60cb)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:04.546448 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b09a36-418d-427f-a4ab-3afc0bf56117 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:04.552976 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 533/1056] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:32:04 2022] GET /identity/v3/users/f94a201f7b784d3e8937d496487e60cb => generated 285 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:32:04.553793 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c55e508-bd05-44ba-b711-91f93facc94d None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:04.615399 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:04.654930 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c55e508-bd05-44ba-b711-91f93facc94d None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:04.688060 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f7623f-0434-416a-a9bb-8b52926535e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:04.688664 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7623f-0434-416a-a9bb-8b52926535e6 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:04.688938 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7623f-0434-416a-a9bb-8b52926535e6 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:04.689205 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7623f-0434-416a-a9bb-8b52926535e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:04.758231 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f7623f-0434-416a-a9bb-8b52926535e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:04.759744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f7623f-0434-416a-a9bb-8b52926535e6 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:04.819979 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:04.822670 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d7e25-944f-437b-af84-742330998e54 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:04.822974 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d7e25-944f-437b-af84-742330998e54 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:04.823245 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d7e25-944f-437b-af84-742330998e54 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:04.828409 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 534/1057] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:32:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 23:32:04.829742 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8edf9-234b-43df-afcc-603d2a30993a None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:04.830019 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8edf9-234b-43df-afcc-603d2a30993a None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:04.830312 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8edf9-234b-43df-afcc-603d2a30993a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:04.849586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-a1a8edf9-234b-43df-afcc-603d2a30993a None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:32:04.850751 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-a1a8edf9-234b-43df-afcc-603d2a30993a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:32:04.869031 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 524/1058] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:32:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 17 23:32:04.869193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:04.870901 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f7623f-0434-416a-a9bb-8b52926535e6 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:04.921628 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 535/1059] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:32:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:32:04.972150 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f7623f-0434-416a-a9bb-8b52926535e6 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:05.006351 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8295e0-30c6-4aac-8658-0460a0cc01b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:05.006969 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8295e0-30c6-4aac-8658-0460a0cc01b3 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:05.007244 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8295e0-30c6-4aac-8658-0460a0cc01b3 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:05.007513 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8295e0-30c6-4aac-8658-0460a0cc01b3 None admin] PATH_INFO: `/v3/projects/7d63c937687247a887936dbef97f25f0` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:05.008812 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8295e0-30c6-4aac-8658-0460a0cc01b3 None admin] RBAC: Authorizing `identity:get_project(project_id=7d63c937687247a887936dbef97f25f0)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:05.010243 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8295e0-30c6-4aac-8658-0460a0cc01b3 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:05.016956 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 525/1060] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:32:04 2022] GET /identity/v3/projects/7d63c937687247a887936dbef97f25f0 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:05.018441 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a8edf9-234b-43df-afcc-603d2a30993a None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:05.121338 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a8edf9-234b-43df-afcc-603d2a30993a None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:05.137924 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae9eda-a56f-456a-95a3-695035bd10c7 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:05.138214 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae9eda-a56f-456a-95a3-695035bd10c7 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:05.138495 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae9eda-a56f-456a-95a3-695035bd10c7 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:05.140009 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 526/1061] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:32:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 23:32:05.144512 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a071d2f-e3b9-4909-ac47-557bfd72c5bd None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:05.144794 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a071d2f-e3b9-4909-ac47-557bfd72c5bd None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:05.145073 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a071d2f-e3b9-4909-ac47-557bfd72c5bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:05.157412 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a09d64-eeed-48f4-91ae-32a83fa1719d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:05.158018 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a09d64-eeed-48f4-91ae-32a83fa1719d None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:05.158423 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a09d64-eeed-48f4-91ae-32a83fa1719d None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:05.158694 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a09d64-eeed-48f4-91ae-32a83fa1719d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:05.159412 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a09d64-eeed-48f4-91ae-32a83fa1719d None admin] RBAC: Adding query filter params (role.id=33bd60d3b0e94dcbbfe97e87f91a8d78, scope.project.id=7d63c937687247a887936dbef97f25f0, user.id=f94a201f7b784d3e8937d496487e60cb) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:32:05.159718 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a09d64-eeed-48f4-91ae-32a83fa1719d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:05.161168 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a09d64-eeed-48f4-91ae-32a83fa1719d None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:05.163633 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-9a071d2f-e3b9-4909-ac47-557bfd72c5bd None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:32:05.165544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-9a071d2f-e3b9-4909-ac47-557bfd72c5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:32:05.172284 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 536/1062] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:32:05 2022] GET /identity/v3/role_assignments?user.id=f94a201f7b784d3e8937d496487e60cb&scope.project.id=7d63c937687247a887936dbef97f25f0&role.id=33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:05.189165 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a09d64-eeed-48f4-91ae-32a83fa1719d None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:05.196128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:05.206366 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:05.248014 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 527/1063] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:32:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 23:32:05.302966 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a09d64-eeed-48f4-91ae-32a83fa1719d None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:05.339537 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64bedee-9620-4655-b80c-b1fb51a9b430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:05.340533 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64bedee-9620-4655-b80c-b1fb51a9b430 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:05.341021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64bedee-9620-4655-b80c-b1fb51a9b430 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:05.341519 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64bedee-9620-4655-b80c-b1fb51a9b430 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:05.408382 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64bedee-9620-4655-b80c-b1fb51a9b430 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:05.410194 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64bedee-9620-4655-b80c-b1fb51a9b430 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:05.426039 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:05.516252 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 537/1064] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:32:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 17 23:32:05.519446 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64bedee-9620-4655-b80c-b1fb51a9b430 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:05.519576 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a071d2f-e3b9-4909-ac47-557bfd72c5bd None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:05.621873 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64bedee-9620-4655-b80c-b1fb51a9b430 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:05.629822 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a071d2f-e3b9-4909-ac47-557bfd72c5bd None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:05.638848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:05.661245 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01b68e-5ff9-41dd-b52e-b837b1d40485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:05.661809 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01b68e-5ff9-41dd-b52e-b837b1d40485 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:05.662092 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01b68e-5ff9-41dd-b52e-b837b1d40485 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:05.662361 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01b68e-5ff9-41dd-b52e-b837b1d40485 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:05.670973 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e36652-df24-4c5d-8f8b-869ad8462f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:05.672266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e36652-df24-4c5d-8f8b-869ad8462f22 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:05.672394 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d01b68e-5ff9-41dd-b52e-b837b1d40485 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:05.672881 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e36652-df24-4c5d-8f8b-869ad8462f22 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:05.673356 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e36652-df24-4c5d-8f8b-869ad8462f22 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:05.673771 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d01b68e-5ff9-41dd-b52e-b837b1d40485 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:05.685570 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e36652-df24-4c5d-8f8b-869ad8462f22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:32:05.688130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-5d01b68e-5ff9-41dd-b52e-b837b1d40485 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:32:05.688130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:32:05.688130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:32:05.688130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:32:05.688130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:32:05.688130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:32:05.688130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:32:05.688130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:32:05.688130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:32:05.688130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:32:05.688130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:32:05.688130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:32:05.688130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:32:05.688130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:32:05.688130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:32:05.688130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:32:05.689087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:32:05.689087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:32:05.689087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:32:05.689087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:32:05.689087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:32:05.689087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:32:05.689087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:32:05.689087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:32:05.689087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:32:05.689087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:32:05.689087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:32:05.689087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:32:05.689087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:32:05.689087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:32:05.689087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:32:05.689087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:32:05.689087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:32:05.695358 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:32:05.695358 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:32:05.695358 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:32:05.695358 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 538/1065] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:32:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:32:05.695652 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-53e36652-df24-4c5d-8f8b-869ad8462f22 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 17 23:32:05.695652 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:32:05.695652 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:32:05.695652 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:32:05.695652 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:32:05.695652 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:32:05.695652 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:32:05.695652 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:32:05.695652 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:32:05.695652 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:32:05.695652 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:32:05.695652 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:32:05.695652 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:32:05.695652 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:32:05.695652 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:32:05.695652 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:32:05.697495 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:32:05.697495 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:32:05.697495 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:32:05.697495 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:32:05.697495 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:32:05.697495 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:32:05.697495 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:32:05.697495 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:32:05.697495 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:32:05.697495 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:32:05.697495 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:32:05.697495 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:32:05.697495 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:32:05.697495 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:32:05.697495 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:32:05.697495 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:32:05.698358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:32:05.698358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:32:05.698358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:32:05.698358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:32:05.698358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:32:05.698358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:32:05.698358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:32:05.698358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:32:05.698358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:32:05.698358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:32:05.698358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 17 23:32:05.698358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:32:05.698358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 528/1066] 10.222.0.26 () {56 vars in 1120 bytes} [Wed Aug 17 23:32:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:32:05.699062 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01b68e-5ff9-41dd-b52e-b837b1d40485 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:05.699134 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e36652-df24-4c5d-8f8b-869ad8462f22 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:05.724236 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:05.805979 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01b68e-5ff9-41dd-b52e-b837b1d40485 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:05.806313 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e36652-df24-4c5d-8f8b-869ad8462f22 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:05.842144 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e05e43e-6604-47fa-98c1-21bc4c49845f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:05.842674 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2814e430-c739-4689-87b3-f9c659955ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:05.842833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05e43e-6604-47fa-98c1-21bc4c49845f None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:05.843016 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05e43e-6604-47fa-98c1-21bc4c49845f None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:05.843272 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05e43e-6604-47fa-98c1-21bc4c49845f None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:05.843388 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2814e430-c739-4689-87b3-f9c659955ddf None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:05.843545 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2814e430-c739-4689-87b3-f9c659955ddf None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:05.843807 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2814e430-c739-4689-87b3-f9c659955ddf None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:05.844000 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e05e43e-6604-47fa-98c1-21bc4c49845f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:32:05.844276 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e05e43e-6604-47fa-98c1-21bc4c49845f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:05.844481 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2814e430-c739-4689-87b3-f9c659955ddf None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:32:05.844780 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2814e430-c739-4689-87b3-f9c659955ddf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:05.845706 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e05e43e-6604-47fa-98c1-21bc4c49845f None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:05.846120 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2814e430-c739-4689-87b3-f9c659955ddf None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:05.854619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 529/1067] 10.222.0.26 () {56 vars in 1115 bytes} [Wed Aug 17 23:32:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:32:05.867990 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 539/1068] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:32:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:05.868721 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e05e43e-6604-47fa-98c1-21bc4c49845f None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:05.960594 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:05.979944 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e05e43e-6604-47fa-98c1-21bc4c49845f None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:06.017074 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04447bd9-109c-4978-b0cf-3080d60dd268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:06.017702 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04447bd9-109c-4978-b0cf-3080d60dd268 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:06.017978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04447bd9-109c-4978-b0cf-3080d60dd268 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:06.018247 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04447bd9-109c-4978-b0cf-3080d60dd268 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:06.029635 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04447bd9-109c-4978-b0cf-3080d60dd268 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:06.031089 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04447bd9-109c-4978-b0cf-3080d60dd268 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:06.045544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-04447bd9-109c-4978-b0cf-3080d60dd268 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:32:06.045544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:32:06.045544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:32:06.045544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:32:06.045544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:32:06.045544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:32:06.045544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:32:06.045544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:32:06.045544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:32:06.045544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:32:06.045544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:32:06.045544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:32:06.045544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:32:06.045544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:32:06.045544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:32:06.045544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:32:06.046710 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:32:06.046710 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:32:06.046710 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:32:06.046710 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:32:06.046710 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:32:06.046710 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:32:06.046710 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:32:06.046710 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:32:06.046710 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:32:06.046710 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:32:06.046710 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:32:06.046710 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:32:06.046710 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:32:06.046710 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:32:06.046710 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:32:06.046710 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:32:06.046710 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:32:06.047833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:32:06.047833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:32:06.047833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:32:06.047833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 530/1069] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:32:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:32:06.053322 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2814e430-c739-4689-87b3-f9c659955ddf None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:06.086354 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:06.167685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2814e430-c739-4689-87b3-f9c659955ddf None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:06.201232 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf43191-8ea0-4b75-b38b-e464bd3d2e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:06.201846 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf43191-8ea0-4b75-b38b-e464bd3d2e5a None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:06.202125 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf43191-8ea0-4b75-b38b-e464bd3d2e5a None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:06.202398 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf43191-8ea0-4b75-b38b-e464bd3d2e5a None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:06.203035 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf43191-8ea0-4b75-b38b-e464bd3d2e5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:32:06.203336 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf43191-8ea0-4b75-b38b-e464bd3d2e5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:06.204723 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf43191-8ea0-4b75-b38b-e464bd3d2e5a None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:06.216234 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 540/1070] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:32:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:32:06.222069 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04447bd9-109c-4978-b0cf-3080d60dd268 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:06.322206 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04447bd9-109c-4978-b0cf-3080d60dd268 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:06.354660 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211b3d61-fb30-4604-beb8-6cb69ca7c98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:06.355274 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b3d61-fb30-4604-beb8-6cb69ca7c98a None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:06.355555 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b3d61-fb30-4604-beb8-6cb69ca7c98a None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:06.355827 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b3d61-fb30-4604-beb8-6cb69ca7c98a None admin] PATH_INFO: `/v3/users/nova` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:06.358484 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:06.363545 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211b3d61-fb30-4604-beb8-6cb69ca7c98a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:06.365148 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211b3d61-fb30-4604-beb8-6cb69ca7c98a None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:06.376138 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-211b3d61-fb30-4604-beb8-6cb69ca7c98a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 17 23:32:06.376138 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:32:06.376138 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:32:06.376138 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:32:06.376138 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:32:06.376138 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:32:06.376138 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:32:06.376138 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:32:06.376138 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:32:06.376138 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:32:06.376138 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:32:06.376138 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:32:06.376138 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 23:32:06.376138 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 23:32:06.376138 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 23:32:06.376138 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 23:32:06.377069 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:32:06.377069 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:32:06.377069 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 23:32:06.377069 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:32:06.377069 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 23:32:06.377069 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:32:06.377069 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:32:06.377069 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:32:06.377069 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:32:06.377069 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:32:06.377069 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:32:06.377069 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:32:06.377069 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:32:06.377069 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:32:06.377069 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:32:06.377069 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:32:06.377069 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:32:06.377069 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:32:06.378426 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:32:06.378426 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:32:06.378426 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 23:32:06.378426 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 23:32:06.378426 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 23:32:06.378426 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 23:32:06.378426 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 23:32:06.378426 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 23:32:06.378426 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 17 23:32:06.378426 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:32:06.378426 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 531/1071] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Aug 17 23:32:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:32:06.383013 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf43191-8ea0-4b75-b38b-e464bd3d2e5a None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:06.471042 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:06.485856 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf43191-8ea0-4b75-b38b-e464bd3d2e5a None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:06.518542 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec56b0-3b08-4551-99d8-ced0f269162f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:06.519150 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec56b0-3b08-4551-99d8-ced0f269162f None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:06.519432 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec56b0-3b08-4551-99d8-ced0f269162f None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:06.519708 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec56b0-3b08-4551-99d8-ced0f269162f None admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:06.520469 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec56b0-3b08-4551-99d8-ced0f269162f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:32:06.520777 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec56b0-3b08-4551-99d8-ced0f269162f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:06.522208 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec56b0-3b08-4551-99d8-ced0f269162f None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:06.540521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 541/1072] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Aug 17 23:32:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:06.546169 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211b3d61-fb30-4604-beb8-6cb69ca7c98a None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:06.644445 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211b3d61-fb30-4604-beb8-6cb69ca7c98a None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:06.676909 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7419d54e-e43e-4a67-9ff0-db698c82a89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:06.677531 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419d54e-e43e-4a67-9ff0-db698c82a89d None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:06.677811 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419d54e-e43e-4a67-9ff0-db698c82a89d None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:06.678079 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419d54e-e43e-4a67-9ff0-db698c82a89d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:06.688802 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7419d54e-e43e-4a67-9ff0-db698c82a89d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:06.690251 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7419d54e-e43e-4a67-9ff0-db698c82a89d None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:06.704503 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-7419d54e-e43e-4a67-9ff0-db698c82a89d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:32:06.704503 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:32:06.704503 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:32:06.704503 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:32:06.704503 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:32:06.704503 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:32:06.704503 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:32:06.704503 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:32:06.704503 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:32:06.704503 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:32:06.704503 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:32:06.704503 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:32:06.704503 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:32:06.704503 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:32:06.704503 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:32:06.704503 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:32:06.705552 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:32:06.705552 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:32:06.705552 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:32:06.705552 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:32:06.705552 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:32:06.705552 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:32:06.705552 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:32:06.705552 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:32:06.705552 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:32:06.705552 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:32:06.705552 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:32:06.705552 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:32:06.705552 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:32:06.705552 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:32:06.705552 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:32:06.705552 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:32:06.705552 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:32:06.706446 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:32:06.706446 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:32:06.706446 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:32:06.706446 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 532/1073] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:32:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:32:06.711613 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec56b0-3b08-4551-99d8-ced0f269162f None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:06.809296 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec56b0-3b08-4551-99d8-ced0f269162f None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:06.842331 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c216f5-efe4-446c-97e9-80ec6a65e87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:06.842922 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c216f5-efe4-446c-97e9-80ec6a65e87a None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:06.843202 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c216f5-efe4-446c-97e9-80ec6a65e87a None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:06.843477 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c216f5-efe4-446c-97e9-80ec6a65e87a None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:06.844149 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c216f5-efe4-446c-97e9-80ec6a65e87a None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:32:06.844453 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c216f5-efe4-446c-97e9-80ec6a65e87a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:06.845945 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c216f5-efe4-446c-97e9-80ec6a65e87a None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:06.857613 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 542/1074] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:32:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:06.863226 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7419d54e-e43e-4a67-9ff0-db698c82a89d None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:06.911478 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:06.962965 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7419d54e-e43e-4a67-9ff0-db698c82a89d None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:06.995794 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c58fe8-efaf-46f0-a5ab-d912b503cf96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:06.996442 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c58fe8-efaf-46f0-a5ab-d912b503cf96 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:06.996720 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c58fe8-efaf-46f0-a5ab-d912b503cf96 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:06.996989 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c58fe8-efaf-46f0-a5ab-d912b503cf96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:07.003472 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c58fe8-efaf-46f0-a5ab-d912b503cf96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:07.004939 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c58fe8-efaf-46f0-a5ab-d912b503cf96 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:07.016533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-15c58fe8-efaf-46f0-a5ab-d912b503cf96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:32:07.016533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:32:07.016533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:32:07.016533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:32:07.016533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:32:07.016533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:32:07.016533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:32:07.016533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:32:07.016533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:32:07.016533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:32:07.016533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:32:07.016533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:32:07.016533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 23:32:07.016533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 23:32:07.016533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 23:32:07.017613 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 23:32:07.017613 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:32:07.017613 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:32:07.017613 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:32:07.017613 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:32:07.017613 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:32:07.017613 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:32:07.017613 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:32:07.017613 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:32:07.017613 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:32:07.017613 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:32:07.017613 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:32:07.017613 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:32:07.017613 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:32:07.017613 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:32:07.017613 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:32:07.017613 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:32:07.018727 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 23:32:07.018727 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 23:32:07.018727 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 23:32:07.018727 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 23:32:07.018727 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 23:32:07.018727 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 23:32:07.018727 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:32:07.018727 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:32:07.020252 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 533/1075] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:32:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:32:07.023488 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c216f5-efe4-446c-97e9-80ec6a65e87a None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:07.028166 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:07.045496 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fbb33-e49a-4474-9094-c82d9036e54e None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:07.045847 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fbb33-e49a-4474-9094-c82d9036e54e None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:07.046131 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fbb33-e49a-4474-9094-c82d9036e54e None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:07.047462 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 534/1076] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:32:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 23:32:07.052291 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe4256-648f-46d6-ac89-0bdb55437fbd None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:07.052573 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe4256-648f-46d6-ac89-0bdb55437fbd None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:07.052848 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe4256-648f-46d6-ac89-0bdb55437fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:07.070745 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-a7fe4256-648f-46d6-ac89-0bdb55437fbd None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:32:07.072442 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-a7fe4256-648f-46d6-ac89-0bdb55437fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:32:07.125160 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c216f5-efe4-446c-97e9-80ec6a65e87a None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:07.139710 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 535/1077] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:32:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 23:32:07.157426 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae38430-a0c5-4775-a5ea-3fa2245b7436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:07.158026 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae38430-a0c5-4775-a5ea-3fa2245b7436 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:07.158310 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae38430-a0c5-4775-a5ea-3fa2245b7436 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:07.158571 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae38430-a0c5-4775-a5ea-3fa2245b7436 None admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:07.159242 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae38430-a0c5-4775-a5ea-3fa2245b7436 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:32:07.159542 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae38430-a0c5-4775-a5ea-3fa2245b7436 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:07.160978 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae38430-a0c5-4775-a5ea-3fa2245b7436 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:07.172820 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 543/1078] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 17 23:32:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:07.178758 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe4256-648f-46d6-ac89-0bdb55437fbd None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:07.217122 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:07.275606 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe4256-648f-46d6-ac89-0bdb55437fbd None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:07.309502 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced2ec4-b63e-406c-83f3-d4c992e10acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:07.310137 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced2ec4-b63e-406c-83f3-d4c992e10acf None admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:07.310411 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced2ec4-b63e-406c-83f3-d4c992e10acf None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:07.310679 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced2ec4-b63e-406c-83f3-d4c992e10acf None admin] PATH_INFO: `/v3/projects/4de86915ef4340248ba8a8f51d56594e/users/ec1dcbe43b0749c3bcc4af9100d771d1/roles/dc16b44096d040dfb737c287986be85e` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:07.312835 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced2ec4-b63e-406c-83f3-d4c992e10acf None admin] RBAC: Authorizing `identity:create_grant(project_id=4de86915ef4340248ba8a8f51d56594e, user_id=ec1dcbe43b0749c3bcc4af9100d771d1, role_id=dc16b44096d040dfb737c287986be85e)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:07.314260 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced2ec4-b63e-406c-83f3-d4c992e10acf None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:07.327270 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 536/1079] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 17 23:32:07 2022] PUT /identity/v3/projects/4de86915ef4340248ba8a8f51d56594e/users/ec1dcbe43b0749c3bcc4af9100d771d1/roles/dc16b44096d040dfb737c287986be85e => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:32:07.408515 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae38430-a0c5-4775-a5ea-3fa2245b7436 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:07.478386 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:07.515253 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae38430-a0c5-4775-a5ea-3fa2245b7436 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:07.548886 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39a599-8dda-46f7-bb7f-988eeb2144d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:07.549510 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39a599-8dda-46f7-bb7f-988eeb2144d8 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:07.549844 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39a599-8dda-46f7-bb7f-988eeb2144d8 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:07.550119 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39a599-8dda-46f7-bb7f-988eeb2144d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:07.559872 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39a599-8dda-46f7-bb7f-988eeb2144d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:32:07.564583 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-4f39a599-8dda-46f7-bb7f-988eeb2144d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:32:07.564583 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:32:07.564583 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:32:07.564583 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:32:07.564583 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:32:07.564583 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:32:07.564583 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:32:07.564583 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:32:07.564583 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:32:07.564583 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:32:07.564583 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:32:07.564583 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:32:07.564583 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:32:07.564583 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:32:07.564583 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:32:07.564583 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:32:07.565678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:32:07.565678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:32:07.565678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:32:07.565678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:32:07.565678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:32:07.565678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:32:07.565678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:32:07.565678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:32:07.565678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:32:07.565678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:32:07.565678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:32:07.565678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:32:07.565678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:32:07.565678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:32:07.565678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:32:07.565678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:32:07.566558 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:32:07.566558 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:32:07.566558 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:32:07.566558 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:32:07.566558 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:32:07.566558 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:32:07.566558 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:32:07.566558 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:32:07.566558 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:32:07.566558 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:32:07.566558 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:32:07.566558 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:32:07.566558 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 544/1080] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 17 23:32:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:32:07.571900 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced2ec4-b63e-406c-83f3-d4c992e10acf None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:07.671379 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:07.675000 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced2ec4-b63e-406c-83f3-d4c992e10acf None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:07.708308 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb306b4-bd14-4801-8c87-433986a69358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:07.708853 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb306b4-bd14-4801-8c87-433986a69358 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:07.709132 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb306b4-bd14-4801-8c87-433986a69358 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:07.709461 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb306b4-bd14-4801-8c87-433986a69358 None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:07.710086 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb306b4-bd14-4801-8c87-433986a69358 None admin] RBAC: Adding query filter params (name=admin) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:32:07.710382 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb306b4-bd14-4801-8c87-433986a69358 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:07.712344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb306b4-bd14-4801-8c87-433986a69358 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:07.721429 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 537/1081] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:32:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:32:07.727235 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39a599-8dda-46f7-bb7f-988eeb2144d8 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:07.824863 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39a599-8dda-46f7-bb7f-988eeb2144d8 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:07.847647 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:07.859133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab86057-0aea-4789-a51b-4abcbb8f685b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:07.859761 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab86057-0aea-4789-a51b-4abcbb8f685b None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:07.860068 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab86057-0aea-4789-a51b-4abcbb8f685b None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:07.860342 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab86057-0aea-4789-a51b-4abcbb8f685b None admin] PATH_INFO: `/v3/users/f94a201f7b784d3e8937d496487e60cb` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:07.861448 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab86057-0aea-4789-a51b-4abcbb8f685b None admin] RBAC: Authorizing `identity:get_user(user_id=f94a201f7b784d3e8937d496487e60cb)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:07.862871 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab86057-0aea-4789-a51b-4abcbb8f685b None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:07.864488 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 545/1082] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:32:07 2022] GET /identity/v3/users/f94a201f7b784d3e8937d496487e60cb => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:07.870290 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb306b4-bd14-4801-8c87-433986a69358 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:07.968960 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb306b4-bd14-4801-8c87-433986a69358 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:08.000973 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d041ce7-45e0-47ca-b34a-992ef3919703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:08.001592 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d041ce7-45e0-47ca-b34a-992ef3919703 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:08.001870 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d041ce7-45e0-47ca-b34a-992ef3919703 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:08.002141 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d041ce7-45e0-47ca-b34a-992ef3919703 None admin] PATH_INFO: `/v3/projects/7d63c937687247a887936dbef97f25f0` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:08.003245 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d041ce7-45e0-47ca-b34a-992ef3919703 None admin] RBAC: Authorizing `identity:get_project(project_id=7d63c937687247a887936dbef97f25f0)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:08.004708 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d041ce7-45e0-47ca-b34a-992ef3919703 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:08.006334 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 538/1083] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:32:07 2022] GET /identity/v3/projects/7d63c937687247a887936dbef97f25f0 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:08.012823 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab86057-0aea-4789-a51b-4abcbb8f685b None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:08.108153 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab86057-0aea-4789-a51b-4abcbb8f685b None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:08.141710 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d2c4c-058b-4cd7-a5f1-6ff0c6579191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:08.142355 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d2c4c-058b-4cd7-a5f1-6ff0c6579191 None admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:08.142633 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d2c4c-058b-4cd7-a5f1-6ff0c6579191 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:08.142902 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d2c4c-058b-4cd7-a5f1-6ff0c6579191 None admin] PATH_INFO: `/v3/projects/7d63c937687247a887936dbef97f25f0/users/f94a201f7b784d3e8937d496487e60cb/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:08.144798 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d2c4c-058b-4cd7-a5f1-6ff0c6579191 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d63c937687247a887936dbef97f25f0, user_id=f94a201f7b784d3e8937d496487e60cb, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:08.146231 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d2c4c-058b-4cd7-a5f1-6ff0c6579191 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:08.159974 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 546/1084] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 17 23:32:08 2022] PUT /identity/v3/projects/7d63c937687247a887936dbef97f25f0/users/f94a201f7b784d3e8937d496487e60cb/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:32:09.204650 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c034bec-8b8c-4da6-96fb-03b83b904a82 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:09.205191 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c034bec-8b8c-4da6-96fb-03b83b904a82 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:09.205191 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c034bec-8b8c-4da6-96fb-03b83b904a82 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:09.206585 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 539/1085] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:32:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:32:09.212840 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7b779-b34b-4994-81eb-61b156e4a389 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:09.213356 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7b779-b34b-4994-81eb-61b156e4a389 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:09.213472 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7b779-b34b-4994-81eb-61b156e4a389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:09.231618 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-f4f7b779-b34b-4994-81eb-61b156e4a389 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:32:09.232833 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-f4f7b779-b34b-4994-81eb-61b156e4a389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:32:09.278474 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:09.298672 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 547/1086] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:32:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:32:09.563439 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c034bec-8b8c-4da6-96fb-03b83b904a82 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:09.595532 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:09.674353 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c034bec-8b8c-4da6-96fb-03b83b904a82 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:09.708812 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e485464-6b0e-4b50-ab21-4d3eb0cde055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:09.709745 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e485464-6b0e-4b50-ab21-4d3eb0cde055 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:09.710173 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e485464-6b0e-4b50-ab21-4d3eb0cde055 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:09.710622 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e485464-6b0e-4b50-ab21-4d3eb0cde055 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:09.722003 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e485464-6b0e-4b50-ab21-4d3eb0cde055 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:32:09.726891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-8e485464-6b0e-4b50-ab21-4d3eb0cde055 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 17 23:32:09.726891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:32:09.726891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:32:09.726891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:32:09.726891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:32:09.726891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:32:09.726891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:32:09.726891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:32:09.726891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:32:09.726891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:32:09.726891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:32:09.726891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:32:09.726891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:32:09.726891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:32:09.726891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:32:09.726891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:32:09.728428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:32:09.728428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:32:09.728428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:32:09.728428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:32:09.728428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:32:09.728428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:32:09.728428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:32:09.728428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:32:09.728428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:32:09.728428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:32:09.728428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:32:09.728428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:32:09.728428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:32:09.728428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:32:09.728428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:32:09.728428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:32:09.729460 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:32:09.729460 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:32:09.729460 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:32:09.729460 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:32:09.729460 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:32:09.729460 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:32:09.729460 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:32:09.729460 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:32:09.729460 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:32:09.729460 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:32:09.729460 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 17 23:32:09.729460 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:32:09.729460 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 540/1087] 10.222.0.26 () {56 vars in 1120 bytes} [Wed Aug 17 23:32:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:32:09.734616 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7b779-b34b-4994-81eb-61b156e4a389 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:09.837428 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7b779-b34b-4994-81eb-61b156e4a389 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:09.871820 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9510f32b-4742-40f4-b53b-245afb0d4cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:09.872461 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510f32b-4742-40f4-b53b-245afb0d4cf7 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:09.872746 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510f32b-4742-40f4-b53b-245afb0d4cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:09.873018 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510f32b-4742-40f4-b53b-245afb0d4cf7 None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:09.873685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9510f32b-4742-40f4-b53b-245afb0d4cf7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:32:09.873989 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9510f32b-4742-40f4-b53b-245afb0d4cf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:09.875363 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9510f32b-4742-40f4-b53b-245afb0d4cf7 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:09.883897 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 548/1088] 10.222.0.26 () {56 vars in 1115 bytes} [Wed Aug 17 23:32:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:09.889782 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e485464-6b0e-4b50-ab21-4d3eb0cde055 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:09.979856 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:09.993529 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e485464-6b0e-4b50-ab21-4d3eb0cde055 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:10.026680 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0652d90-dafd-485b-b0a4-314931459d6e None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:10.026963 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0652d90-dafd-485b-b0a4-314931459d6e None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:10.027237 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0652d90-dafd-485b-b0a4-314931459d6e None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:10.028303 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f8028-db57-418e-90ef-d9478018130e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:10.028857 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 549/1089] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:32:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 23:32:10.028957 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f8028-db57-418e-90ef-d9478018130e None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:10.029261 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f8028-db57-418e-90ef-d9478018130e None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:10.029533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f8028-db57-418e-90ef-d9478018130e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:10.033521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8c6dc-5ea4-4e93-9d59-1cf9043e454d None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:10.033803 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8c6dc-5ea4-4e93-9d59-1cf9043e454d None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:10.034073 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8c6dc-5ea4-4e93-9d59-1cf9043e454d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:10.039787 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610f8028-db57-418e-90ef-d9478018130e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:10.042354 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610f8028-db57-418e-90ef-d9478018130e None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:10.051449 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-5fa8c6dc-5ea4-4e93-9d59-1cf9043e454d None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:32:10.052709 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-5fa8c6dc-5ea4-4e93-9d59-1cf9043e454d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:32:10.057814 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-610f8028-db57-418e-90ef-d9478018130e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:32:10.057814 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:32:10.057814 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:32:10.057814 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:32:10.057814 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:32:10.057814 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:32:10.057814 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:32:10.057814 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:32:10.057814 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:32:10.057814 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:32:10.057814 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:32:10.057814 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:32:10.057814 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:32:10.057814 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:32:10.057814 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:32:10.057814 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:32:10.058934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:32:10.058934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:32:10.058934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:32:10.058934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:32:10.058934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:32:10.058934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:32:10.058934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:32:10.058934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:32:10.058934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:32:10.058934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:32:10.058934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:32:10.058934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:32:10.058934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:32:10.058934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:32:10.058934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:32:10.058934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:32:10.058934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:32:10.060056 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:32:10.060056 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:32:10.060056 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:32:10.064179 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 541/1090] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:32:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:32:10.064622 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f8028-db57-418e-90ef-d9478018130e None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:10.093485 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:10.120917 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 550/1091] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:32:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:32:10.163195 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f8028-db57-418e-90ef-d9478018130e None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:10.196141 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d54a2d1-6033-43fe-bda0-f3248d78b61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:10.196736 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54a2d1-6033-43fe-bda0-f3248d78b61f None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:10.197018 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54a2d1-6033-43fe-bda0-f3248d78b61f None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:10.197293 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54a2d1-6033-43fe-bda0-f3248d78b61f None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:10.197934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d54a2d1-6033-43fe-bda0-f3248d78b61f None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:32:10.198237 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d54a2d1-6033-43fe-bda0-f3248d78b61f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:10.199599 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d54a2d1-6033-43fe-bda0-f3248d78b61f None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:10.208078 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:10.212115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 542/1092] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:32:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:10.217325 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8c6dc-5ea4-4e93-9d59-1cf9043e454d None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:10.313073 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8c6dc-5ea4-4e93-9d59-1cf9043e454d None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:10.345085 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceaf651-d0fc-43ff-b085-cfff56efc275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:10.345702 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceaf651-d0fc-43ff-b085-cfff56efc275 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:10.345986 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceaf651-d0fc-43ff-b085-cfff56efc275 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:10.346260 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceaf651-d0fc-43ff-b085-cfff56efc275 None admin] PATH_INFO: `/v3/users/nova` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:10.353450 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceaf651-d0fc-43ff-b085-cfff56efc275 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:10.354887 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceaf651-d0fc-43ff-b085-cfff56efc275 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:10.366100 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-3ceaf651-d0fc-43ff-b085-cfff56efc275 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 17 23:32:10.366100 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:32:10.366100 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:32:10.366100 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:32:10.366100 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:32:10.366100 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:32:10.366100 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:32:10.366100 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:32:10.366100 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:32:10.366100 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:32:10.366100 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:32:10.366100 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:32:10.366100 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 23:32:10.366100 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 23:32:10.366100 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 23:32:10.366100 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 23:32:10.367020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:32:10.367020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:32:10.367020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 23:32:10.367020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:32:10.367020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 23:32:10.367020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:32:10.367020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:32:10.367020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:32:10.367020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:32:10.367020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:32:10.367020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:32:10.367020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:32:10.367020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:32:10.367020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:32:10.367020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:32:10.367020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:32:10.367020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:32:10.367020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:32:10.368615 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:32:10.368615 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:32:10.368615 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 23:32:10.368615 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 23:32:10.368615 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 23:32:10.368615 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 23:32:10.368615 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 23:32:10.368615 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 23:32:10.368615 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 17 23:32:10.368615 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:32:10.368615 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 551/1093] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Aug 17 23:32:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:32:10.373441 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d54a2d1-6033-43fe-bda0-f3248d78b61f None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:10.388470 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceaf651-d0fc-43ff-b085-cfff56efc275 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:10.423002 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:10.475220 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d54a2d1-6033-43fe-bda0-f3248d78b61f None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:10.490097 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceaf651-d0fc-43ff-b085-cfff56efc275 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:10.509716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a08bcd7-e081-4702-a423-8443ea0c1bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:10.510314 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08bcd7-e081-4702-a423-8443ea0c1bad None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:10.510586 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08bcd7-e081-4702-a423-8443ea0c1bad None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:10.510851 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08bcd7-e081-4702-a423-8443ea0c1bad None admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:10.511569 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a08bcd7-e081-4702-a423-8443ea0c1bad None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:32:10.511869 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a08bcd7-e081-4702-a423-8443ea0c1bad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:10.513338 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a08bcd7-e081-4702-a423-8443ea0c1bad None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:10.523382 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c75b06-503a-4cc7-ad19-d5b2a79d250b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:10.524055 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c75b06-503a-4cc7-ad19-d5b2a79d250b None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:10.524338 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c75b06-503a-4cc7-ad19-d5b2a79d250b None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:10.524614 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c75b06-503a-4cc7-ad19-d5b2a79d250b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:10.529958 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 543/1094] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Aug 17 23:32:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:10.535680 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a08bcd7-e081-4702-a423-8443ea0c1bad None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:10.535909 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c75b06-503a-4cc7-ad19-d5b2a79d250b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:32:10.540642 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-02c75b06-503a-4cc7-ad19-d5b2a79d250b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:32:10.540642 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:32:10.540642 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:32:10.540642 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:32:10.540642 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:32:10.540642 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:32:10.540642 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:32:10.540642 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:32:10.540642 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:32:10.540642 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:32:10.540642 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:32:10.540642 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:32:10.540642 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:32:10.540642 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:32:10.540642 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:32:10.540642 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:32:10.541685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:32:10.541685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:32:10.541685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:32:10.541685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:32:10.541685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:32:10.541685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:32:10.541685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:32:10.541685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:32:10.541685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:32:10.541685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:32:10.541685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:32:10.541685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:32:10.541685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:32:10.541685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:32:10.541685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:32:10.541685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:32:10.542559 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:32:10.542559 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:32:10.542559 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:32:10.542559 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:32:10.542559 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:32:10.542559 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:32:10.542559 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:32:10.542559 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:32:10.542559 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:32:10.542559 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:32:10.542559 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:32:10.542559 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:32:10.542559 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 552/1095] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 17 23:32:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:32:10.547562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c75b06-503a-4cc7-ad19-d5b2a79d250b None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:10.592928 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:10.632644 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a08bcd7-e081-4702-a423-8443ea0c1bad None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:10.637824 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:10.645214 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c75b06-503a-4cc7-ad19-d5b2a79d250b None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:10.666074 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c613b-64cb-4786-9cb3-add12922dee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:10.666668 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c613b-64cb-4786-9cb3-add12922dee1 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:10.666945 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c613b-64cb-4786-9cb3-add12922dee1 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:10.667213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c613b-64cb-4786-9cb3-add12922dee1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:10.678473 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc278b5-1999-42b7-910d-29e21fa95850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:10.679075 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc278b5-1999-42b7-910d-29e21fa95850 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:10.679356 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc278b5-1999-42b7-910d-29e21fa95850 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:10.679631 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc278b5-1999-42b7-910d-29e21fa95850 None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:10.680845 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc278b5-1999-42b7-910d-29e21fa95850 None admin] RBAC: Adding query filter params (name=admin) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:32:10.681150 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc278b5-1999-42b7-910d-29e21fa95850 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:10.682514 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc278b5-1999-42b7-910d-29e21fa95850 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:10.690899 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 553/1096] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:32:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:32:10.697262 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc278b5-1999-42b7-910d-29e21fa95850 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:10.733672 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778c613b-64cb-4786-9cb3-add12922dee1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:10.735122 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778c613b-64cb-4786-9cb3-add12922dee1 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:10.790264 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:10.792369 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc278b5-1999-42b7-910d-29e21fa95850 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:10.826100 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3aa996-ad22-40d4-b9e5-7049a2f62fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:10.826716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3aa996-ad22-40d4-b9e5-7049a2f62fc4 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:10.826990 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3aa996-ad22-40d4-b9e5-7049a2f62fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:10.827264 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3aa996-ad22-40d4-b9e5-7049a2f62fc4 None admin] PATH_INFO: `/v3/users/f94a201f7b784d3e8937d496487e60cb` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:10.828407 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3aa996-ad22-40d4-b9e5-7049a2f62fc4 None admin] RBAC: Authorizing `identity:get_user(user_id=f94a201f7b784d3e8937d496487e60cb)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:10.829809 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3aa996-ad22-40d4-b9e5-7049a2f62fc4 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:10.831361 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 554/1097] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:32:10 2022] GET /identity/v3/users/f94a201f7b784d3e8937d496487e60cb => generated 285 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:32:10.833690 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 544/1098] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:32:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 23:32:10.837457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3aa996-ad22-40d4-b9e5-7049a2f62fc4 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:10.839109 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c613b-64cb-4786-9cb3-add12922dee1 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:10.862112 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:10.936599 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3aa996-ad22-40d4-b9e5-7049a2f62fc4 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:10.937089 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c613b-64cb-4786-9cb3-add12922dee1 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:10.969832 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a099744-6318-4f14-8811-ea93ab1b0820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:10.970404 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8527f0-8652-4b72-95ee-4bf8f2f06602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:10.970527 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a099744-6318-4f14-8811-ea93ab1b0820 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:10.970712 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a099744-6318-4f14-8811-ea93ab1b0820 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:10.971012 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a099744-6318-4f14-8811-ea93ab1b0820 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:10.971119 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8527f0-8652-4b72-95ee-4bf8f2f06602 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:10.971294 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8527f0-8652-4b72-95ee-4bf8f2f06602 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:10.971568 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8527f0-8652-4b72-95ee-4bf8f2f06602 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:10.981904 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8527f0-8652-4b72-95ee-4bf8f2f06602 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:10.983349 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8527f0-8652-4b72-95ee-4bf8f2f06602 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:10.998127 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-6a8527f0-8652-4b72-95ee-4bf8f2f06602 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:32:10.998127 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:32:10.998127 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:32:10.998127 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:32:10.998127 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:32:10.998127 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:32:10.998127 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:32:10.998127 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:32:10.998127 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:32:10.998127 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:32:10.998127 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:32:10.998127 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:32:10.998127 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:32:10.998127 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:32:10.998127 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:32:10.998127 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:32:10.999109 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:32:10.999109 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:32:10.999109 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:32:10.999109 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:32:10.999109 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:32:10.999109 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:32:10.999109 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:32:10.999109 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:32:10.999109 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:32:10.999109 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:32:10.999109 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:32:10.999109 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:32:10.999109 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:32:10.999109 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:32:10.999109 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:32:10.999109 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:32:10.999109 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:32:11.000263 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:32:11.000263 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:32:11.000263 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:32:11.000263 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 545/1099] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:32:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:32:11.004810 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8527f0-8652-4b72-95ee-4bf8f2f06602 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:11.017799 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:11.040721 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a099744-6318-4f14-8811-ea93ab1b0820 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:11.042172 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a099744-6318-4f14-8811-ea93ab1b0820 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:11.064858 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:11.106960 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8527f0-8652-4b72-95ee-4bf8f2f06602 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:11.139973 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 555/1100] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:32:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 17 23:32:11.140852 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887ee9c-48c2-444e-8398-84857a9a8ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:11.141530 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887ee9c-48c2-444e-8398-84857a9a8ab0 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:11.141816 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887ee9c-48c2-444e-8398-84857a9a8ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:11.142089 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887ee9c-48c2-444e-8398-84857a9a8ab0 None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:11.142735 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6887ee9c-48c2-444e-8398-84857a9a8ab0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:32:11.143037 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6887ee9c-48c2-444e-8398-84857a9a8ab0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:11.144475 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6887ee9c-48c2-444e-8398-84857a9a8ab0 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:11.145826 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a099744-6318-4f14-8811-ea93ab1b0820 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:11.157028 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 546/1101] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:32:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:32:11.162286 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887ee9c-48c2-444e-8398-84857a9a8ab0 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:11.228966 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:11.240142 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a099744-6318-4f14-8811-ea93ab1b0820 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:11.260348 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887ee9c-48c2-444e-8398-84857a9a8ab0 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:11.264789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:11.275800 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e9251c-bc20-4e38-a940-037a7ed6c5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:11.276532 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9251c-bc20-4e38-a940-037a7ed6c5d8 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:11.276811 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9251c-bc20-4e38-a940-037a7ed6c5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:11.277083 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9251c-bc20-4e38-a940-037a7ed6c5d8 None admin] PATH_INFO: `/v3/projects/7d63c937687247a887936dbef97f25f0` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:11.278463 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e9251c-bc20-4e38-a940-037a7ed6c5d8 None admin] RBAC: Authorizing `identity:get_project(project_id=7d63c937687247a887936dbef97f25f0)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:11.279891 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e9251c-bc20-4e38-a940-037a7ed6c5d8 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:11.281541 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 556/1102] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:32:11 2022] GET /identity/v3/projects/7d63c937687247a887936dbef97f25f0 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:32:11.287454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e9251c-bc20-4e38-a940-037a7ed6c5d8 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:11.296211 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1cc5e-5438-49ad-ab2f-bea33e82b134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:11.296822 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1cc5e-5438-49ad-ab2f-bea33e82b134 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:11.297108 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1cc5e-5438-49ad-ab2f-bea33e82b134 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:11.297737 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1cc5e-5438-49ad-ab2f-bea33e82b134 None admin] PATH_INFO: `/v3/projects/service` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:11.304676 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1cc5e-5438-49ad-ab2f-bea33e82b134 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:11.306617 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1cc5e-5438-49ad-ab2f-bea33e82b134 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:11.317607 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-5ec1cc5e-5438-49ad-ab2f-bea33e82b134 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:32:11.317607 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:32:11.317607 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:32:11.317607 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:32:11.317607 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:32:11.317607 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:32:11.317607 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:32:11.317607 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:32:11.317607 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:32:11.317607 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:32:11.317607 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:32:11.317607 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:32:11.317607 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 23:32:11.317607 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 23:32:11.317607 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 23:32:11.318515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 23:32:11.318515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:32:11.318515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:32:11.318515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:32:11.318515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:32:11.318515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:32:11.318515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:32:11.318515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:32:11.318515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:32:11.318515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:32:11.318515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:32:11.318515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:32:11.318515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:32:11.318515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:32:11.318515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:32:11.318515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:32:11.318515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:32:11.319443 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 23:32:11.319443 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 23:32:11.319443 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 23:32:11.319443 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 23:32:11.319443 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 23:32:11.319443 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 23:32:11.319443 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:32:11.319443 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:32:11.319443 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 547/1103] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:32:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:32:11.324897 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1cc5e-5438-49ad-ab2f-bea33e82b134 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:11.384433 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e9251c-bc20-4e38-a940-037a7ed6c5d8 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:11.417146 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b13e5-8e96-42cd-ada7-f794e207a0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:11.417758 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b13e5-8e96-42cd-ada7-f794e207a0f7 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:11.418039 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b13e5-8e96-42cd-ada7-f794e207a0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:11.418317 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b13e5-8e96-42cd-ada7-f794e207a0f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:11.419026 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b13e5-8e96-42cd-ada7-f794e207a0f7 None admin] RBAC: Adding query filter params (role.id=33bd60d3b0e94dcbbfe97e87f91a8d78, scope.project.id=7d63c937687247a887936dbef97f25f0, user.id=f94a201f7b784d3e8937d496487e60cb) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:32:11.419330 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b13e5-8e96-42cd-ada7-f794e207a0f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:11.420776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b13e5-8e96-42cd-ada7-f794e207a0f7 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:11.421006 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1cc5e-5438-49ad-ab2f-bea33e82b134 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:11.427185 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 557/1104] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:32:11 2022] GET /identity/v3/role_assignments?user.id=f94a201f7b784d3e8937d496487e60cb&scope.project.id=7d63c937687247a887936dbef97f25f0&role.id=33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:32:11.454500 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:11.461395 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a28042-cedf-4fe5-b8fc-955c81d59d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:11.462041 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a28042-cedf-4fe5-b8fc-955c81d59d89 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:11.462319 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a28042-cedf-4fe5-b8fc-955c81d59d89 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:11.462588 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a28042-cedf-4fe5-b8fc-955c81d59d89 None admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:11.463300 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a28042-cedf-4fe5-b8fc-955c81d59d89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:32:11.463607 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a28042-cedf-4fe5-b8fc-955c81d59d89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:11.465111 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a28042-cedf-4fe5-b8fc-955c81d59d89 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:11.479194 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 548/1105] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 17 23:32:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:32:11.485373 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b13e5-8e96-42cd-ada7-f794e207a0f7 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:11.544695 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:11.591193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b13e5-8e96-42cd-ada7-f794e207a0f7 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:11.624912 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639f8a5-ed67-4924-94ab-d5dc8d8bc055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:11.625511 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639f8a5-ed67-4924-94ab-d5dc8d8bc055 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:11.625788 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639f8a5-ed67-4924-94ab-d5dc8d8bc055 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:11.626060 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639f8a5-ed67-4924-94ab-d5dc8d8bc055 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:11.626778 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5639f8a5-ed67-4924-94ab-d5dc8d8bc055 None admin] RBAC: Adding query filter params (role.id=dc16b44096d040dfb737c287986be85e, scope.project.id=4de86915ef4340248ba8a8f51d56594e, user.id=ec1dcbe43b0749c3bcc4af9100d771d1) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:32:11.627086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5639f8a5-ed67-4924-94ab-d5dc8d8bc055 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:11.628546 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5639f8a5-ed67-4924-94ab-d5dc8d8bc055 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:11.639988 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 558/1106] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:32:11 2022] GET /identity/v3/role_assignments?user.id=ec1dcbe43b0749c3bcc4af9100d771d1&scope.project.id=4de86915ef4340248ba8a8f51d56594e&role.id=dc16b44096d040dfb737c287986be85e => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:13.336752 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25328b-7a18-47a5-b0db-36fa84588881 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:13.337281 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25328b-7a18-47a5-b0db-36fa84588881 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:13.337281 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25328b-7a18-47a5-b0db-36fa84588881 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:13.339215 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 549/1107] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:32:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:32:13.344218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa976bd7-4127-4521-89cb-4edd2e64825a None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:13.344713 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa976bd7-4127-4521-89cb-4edd2e64825a None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:13.344809 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa976bd7-4127-4521-89cb-4edd2e64825a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:13.362079 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-aa976bd7-4127-4521-89cb-4edd2e64825a None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:32:13.363244 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-aa976bd7-4127-4521-89cb-4edd2e64825a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:32:13.427012 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 559/1108] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:32:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:32:13.699052 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e25328b-7a18-47a5-b0db-36fa84588881 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:13.803215 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e25328b-7a18-47a5-b0db-36fa84588881 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:13.835479 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e55e3f2-ff86-4e18-b146-09374fae7518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:13.836127 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55e3f2-ff86-4e18-b146-09374fae7518 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:13.837031 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55e3f2-ff86-4e18-b146-09374fae7518 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:13.837310 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55e3f2-ff86-4e18-b146-09374fae7518 None admin] PATH_INFO: `/v3/domains/default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:13.838426 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e55e3f2-ff86-4e18-b146-09374fae7518 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:13.839863 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e55e3f2-ff86-4e18-b146-09374fae7518 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:13.841448 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 550/1109] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:32:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:13.847196 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa976bd7-4127-4521-89cb-4edd2e64825a None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:13.893964 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:13.941494 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa976bd7-4127-4521-89cb-4edd2e64825a None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:13.974207 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c77bac-de67-40d7-a10a-a14b65f43617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:13.974805 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c77bac-de67-40d7-a10a-a14b65f43617 None admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:13.975081 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c77bac-de67-40d7-a10a-a14b65f43617 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:13.975349 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c77bac-de67-40d7-a10a-a14b65f43617 None admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:13.976022 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c77bac-de67-40d7-a10a-a14b65f43617 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:13.977487 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c77bac-de67-40d7-a10a-a14b65f43617 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:13.997673 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 560/1110] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Aug 17 23:32:13 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:32:15.884682 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fe7d5-ef03-49d6-aa9e-f7c65956a256 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:15.885207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fe7d5-ef03-49d6-aa9e-f7c65956a256 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:15.885207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fe7d5-ef03-49d6-aa9e-f7c65956a256 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:15.886569 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 551/1111] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:32:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:32:15.891751 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee02c4-1fe8-4dca-ab17-a67e059c1d4c None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:15.892210 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee02c4-1fe8-4dca-ab17-a67e059c1d4c None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:15.892309 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee02c4-1fe8-4dca-ab17-a67e059c1d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:15.910080 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-75ee02c4-1fe8-4dca-ab17-a67e059c1d4c None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:32:15.911251 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-75ee02c4-1fe8-4dca-ab17-a67e059c1d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:32:15.976831 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 561/1112] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:32:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 23:32:16.240279 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5fe7d5-ef03-49d6-aa9e-f7c65956a256 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:16.298434 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:16.339587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5fe7d5-ef03-49d6-aa9e-f7c65956a256 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:16.372263 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d35f62-0e05-4cc8-a3fc-3c0664638b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:16.372872 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d35f62-0e05-4cc8-a3fc-3c0664638b38 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:16.373151 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d35f62-0e05-4cc8-a3fc-3c0664638b38 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:16.373424 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d35f62-0e05-4cc8-a3fc-3c0664638b38 None admin] PATH_INFO: `/v3/roles/3428ff8e92274de6817b7e8ee9a56d13` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:16.374797 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d35f62-0e05-4cc8-a3fc-3c0664638b38 None admin] RBAC: Authorizing `identity:get_role(role_id=3428ff8e92274de6817b7e8ee9a56d13)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:16.376261 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d35f62-0e05-4cc8-a3fc-3c0664638b38 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:16.377565 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 552/1113] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:32:16 2022] GET /identity/v3/roles/3428ff8e92274de6817b7e8ee9a56d13 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:16.383830 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee02c4-1fe8-4dca-ab17-a67e059c1d4c None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:16.416635 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:16.480959 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee02c4-1fe8-4dca-ab17-a67e059c1d4c None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:16.514989 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d73cbd-34d9-4806-be84-64e1013489da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:16.515552 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d73cbd-34d9-4806-be84-64e1013489da None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:16.515830 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d73cbd-34d9-4806-be84-64e1013489da None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:16.516131 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d73cbd-34d9-4806-be84-64e1013489da None admin] PATH_INFO: `/v3/users/493489a2b3de475daa9d885dfb6f1bd7` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:16.528904 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d73cbd-34d9-4806-be84-64e1013489da None admin] RBAC: Authorizing `identity:get_user(user_id=493489a2b3de475daa9d885dfb6f1bd7)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:16.530331 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d73cbd-34d9-4806-be84-64e1013489da None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:16.531904 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 562/1114] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:32:16 2022] GET /identity/v3/users/493489a2b3de475daa9d885dfb6f1bd7 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:16.537666 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d35f62-0e05-4cc8-a3fc-3c0664638b38 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:16.635737 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d35f62-0e05-4cc8-a3fc-3c0664638b38 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:16.659977 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:16.668914 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248c468-2411-4862-bd50-749ce507864e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:16.669502 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248c468-2411-4862-bd50-749ce507864e None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:16.669781 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248c468-2411-4862-bd50-749ce507864e None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:16.670051 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248c468-2411-4862-bd50-749ce507864e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:16.737144 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8248c468-2411-4862-bd50-749ce507864e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:16.738590 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8248c468-2411-4862-bd50-749ce507864e None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:16.833222 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 553/1115] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:32:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 23:32:16.839075 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d73cbd-34d9-4806-be84-64e1013489da None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:16.928123 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:16.935804 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d73cbd-34d9-4806-be84-64e1013489da None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:16.967524 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b87fbd2-9daa-42d7-8910-c926b00be44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:16.968378 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87fbd2-9daa-42d7-8910-c926b00be44d None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:16.968659 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87fbd2-9daa-42d7-8910-c926b00be44d None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:16.968929 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87fbd2-9daa-42d7-8910-c926b00be44d None admin] PATH_INFO: `/v3/projects/7d63c937687247a887936dbef97f25f0` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:16.970035 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b87fbd2-9daa-42d7-8910-c926b00be44d None admin] RBAC: Authorizing `identity:get_project(project_id=7d63c937687247a887936dbef97f25f0)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:16.971446 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b87fbd2-9daa-42d7-8910-c926b00be44d None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:16.973075 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 563/1116] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:32:16 2022] GET /identity/v3/projects/7d63c937687247a887936dbef97f25f0 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:16.979744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248c468-2411-4862-bd50-749ce507864e None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:17.006055 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:17.076106 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248c468-2411-4862-bd50-749ce507864e None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:17.108736 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7a8954-b95f-4234-803d-03b8d3a66f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:17.109268 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a8954-b95f-4234-803d-03b8d3a66f09 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:17.109547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a8954-b95f-4234-803d-03b8d3a66f09 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:17.109818 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a8954-b95f-4234-803d-03b8d3a66f09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:17.110536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7a8954-b95f-4234-803d-03b8d3a66f09 None admin] RBAC: Adding query filter params (role.id=3428ff8e92274de6817b7e8ee9a56d13, scope.project.id=7d63c937687247a887936dbef97f25f0, user.id=493489a2b3de475daa9d885dfb6f1bd7) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:32:17.110845 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7a8954-b95f-4234-803d-03b8d3a66f09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:17.112303 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7a8954-b95f-4234-803d-03b8d3a66f09 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:17.117515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 554/1117] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:32:16 2022] GET /identity/v3/role_assignments?user.id=493489a2b3de475daa9d885dfb6f1bd7&scope.project.id=7d63c937687247a887936dbef97f25f0&role.id=3428ff8e92274de6817b7e8ee9a56d13 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:18.944730 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1e423-627d-44be-9bbc-b1234722ac99 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:18.945270 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1e423-627d-44be-9bbc-b1234722ac99 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:18.945270 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1e423-627d-44be-9bbc-b1234722ac99 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:18.947199 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 564/1118] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:32:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:32:18.952406 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7f542-6b1e-42b9-95ce-df38fab3a23f None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:18.952795 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7f542-6b1e-42b9-95ce-df38fab3a23f None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:18.952956 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7f542-6b1e-42b9-95ce-df38fab3a23f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:18.970571 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-05f7f542-6b1e-42b9-95ce-df38fab3a23f None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:32:18.971747 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-05f7f542-6b1e-42b9-95ce-df38fab3a23f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:32:19.037871 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 555/1119] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:32:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 23:32:19.329286 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e1e423-627d-44be-9bbc-b1234722ac99 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:19.427987 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e1e423-627d-44be-9bbc-b1234722ac99 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:19.462307 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a114d-4423-4ccb-b203-74be6366766d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:19.462933 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a114d-4423-4ccb-b203-74be6366766d None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:19.463209 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a114d-4423-4ccb-b203-74be6366766d None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:19.463477 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a114d-4423-4ccb-b203-74be6366766d None admin] PATH_INFO: `/v3/roles/3428ff8e92274de6817b7e8ee9a56d13` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:19.464881 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82a114d-4423-4ccb-b203-74be6366766d None admin] RBAC: Authorizing `identity:get_role(role_id=3428ff8e92274de6817b7e8ee9a56d13)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:19.466241 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82a114d-4423-4ccb-b203-74be6366766d None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:19.467556 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 565/1120] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:32:19 2022] GET /identity/v3/roles/3428ff8e92274de6817b7e8ee9a56d13 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:19.473930 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f7f542-6b1e-42b9-95ce-df38fab3a23f None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:19.494709 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:19.570396 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f7f542-6b1e-42b9-95ce-df38fab3a23f None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:19.608953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cca01b-cd2a-4e90-9c8e-d2f7b66def4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:19.609497 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cca01b-cd2a-4e90-9c8e-d2f7b66def4e None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:19.609771 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cca01b-cd2a-4e90-9c8e-d2f7b66def4e None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:19.610037 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cca01b-cd2a-4e90-9c8e-d2f7b66def4e None admin] PATH_INFO: `/v3/users/493489a2b3de475daa9d885dfb6f1bd7` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:19.611140 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cca01b-cd2a-4e90-9c8e-d2f7b66def4e None admin] RBAC: Authorizing `identity:get_user(user_id=493489a2b3de475daa9d885dfb6f1bd7)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:19.612603 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cca01b-cd2a-4e90-9c8e-d2f7b66def4e None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:19.614175 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 556/1121] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:32:19 2022] GET /identity/v3/users/493489a2b3de475daa9d885dfb6f1bd7 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:19.620537 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a114d-4423-4ccb-b203-74be6366766d None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:19.650418 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:19.717901 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a114d-4423-4ccb-b203-74be6366766d None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:19.750896 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc842447-ff95-48f2-a605-0a3864dba30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:19.751510 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc842447-ff95-48f2-a605-0a3864dba30c None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:19.751784 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc842447-ff95-48f2-a605-0a3864dba30c None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:19.752085 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc842447-ff95-48f2-a605-0a3864dba30c None admin] PATH_INFO: `/v3/projects/7d63c937687247a887936dbef97f25f0` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:19.753196 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc842447-ff95-48f2-a605-0a3864dba30c None admin] RBAC: Authorizing `identity:get_project(project_id=7d63c937687247a887936dbef97f25f0)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:19.754618 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc842447-ff95-48f2-a605-0a3864dba30c None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:19.756245 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 566/1122] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:32:19 2022] GET /identity/v3/projects/7d63c937687247a887936dbef97f25f0 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:19.762218 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cca01b-cd2a-4e90-9c8e-d2f7b66def4e None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:19.847212 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:19.859404 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cca01b-cd2a-4e90-9c8e-d2f7b66def4e None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:19.892052 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe4b07-9fe7-41cc-88b2-aef54c8b16fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:19.892689 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe4b07-9fe7-41cc-88b2-aef54c8b16fb None admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:19.892965 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe4b07-9fe7-41cc-88b2-aef54c8b16fb None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:19.893235 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe4b07-9fe7-41cc-88b2-aef54c8b16fb None admin] PATH_INFO: `/v3/projects/7d63c937687247a887936dbef97f25f0/users/493489a2b3de475daa9d885dfb6f1bd7/roles/3428ff8e92274de6817b7e8ee9a56d13` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:19.895327 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efe4b07-9fe7-41cc-88b2-aef54c8b16fb None admin] RBAC: Authorizing `identity:create_grant(project_id=7d63c937687247a887936dbef97f25f0, user_id=493489a2b3de475daa9d885dfb6f1bd7, role_id=3428ff8e92274de6817b7e8ee9a56d13)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:19.896781 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efe4b07-9fe7-41cc-88b2-aef54c8b16fb None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:19.908674 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 557/1123] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 17 23:32:19 2022] PUT /identity/v3/projects/7d63c937687247a887936dbef97f25f0/users/493489a2b3de475daa9d885dfb6f1bd7/roles/3428ff8e92274de6817b7e8ee9a56d13 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:32:21.757952 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72e1a7-9bb1-4310-800b-c7ea18d03665 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:21.758968 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72e1a7-9bb1-4310-800b-c7ea18d03665 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:21.759098 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72e1a7-9bb1-4310-800b-c7ea18d03665 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:21.760500 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 567/1124] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:32:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:32:21.765471 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2e3f5-1026-42b5-85a5-3cf20f52eed7 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:21.765881 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2e3f5-1026-42b5-85a5-3cf20f52eed7 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:21.766016 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2e3f5-1026-42b5-85a5-3cf20f52eed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:21.783713 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-b8a2e3f5-1026-42b5-85a5-3cf20f52eed7 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:32:21.784917 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-b8a2e3f5-1026-42b5-85a5-3cf20f52eed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:32:21.847797 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 558/1125] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:32:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 23:32:22.115121 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72e1a7-9bb1-4310-800b-c7ea18d03665 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:22.208832 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:22.214417 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72e1a7-9bb1-4310-800b-c7ea18d03665 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:22.246670 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf34ad-a497-42c7-bb7a-b4903ff46542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:22.247286 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf34ad-a497-42c7-bb7a-b4903ff46542 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:22.247566 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf34ad-a497-42c7-bb7a-b4903ff46542 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:22.247839 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf34ad-a497-42c7-bb7a-b4903ff46542 None admin] PATH_INFO: `/v3/roles/3428ff8e92274de6817b7e8ee9a56d13` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:22.249238 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bf34ad-a497-42c7-bb7a-b4903ff46542 None admin] RBAC: Authorizing `identity:get_role(role_id=3428ff8e92274de6817b7e8ee9a56d13)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:22.250595 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bf34ad-a497-42c7-bb7a-b4903ff46542 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:22.251882 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 568/1126] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:32:22 2022] GET /identity/v3/roles/3428ff8e92274de6817b7e8ee9a56d13 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:22.257622 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a2e3f5-1026-42b5-85a5-3cf20f52eed7 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:22.332848 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:22.354598 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a2e3f5-1026-42b5-85a5-3cf20f52eed7 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:22.386999 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3515a81a-9aba-4752-ab2b-f7d68017f8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:22.387561 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515a81a-9aba-4752-ab2b-f7d68017f8f0 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:22.387838 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515a81a-9aba-4752-ab2b-f7d68017f8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:22.388136 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515a81a-9aba-4752-ab2b-f7d68017f8f0 None admin] PATH_INFO: `/v3/users/493489a2b3de475daa9d885dfb6f1bd7` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:22.389239 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3515a81a-9aba-4752-ab2b-f7d68017f8f0 None admin] RBAC: Authorizing `identity:get_user(user_id=493489a2b3de475daa9d885dfb6f1bd7)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:22.390666 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3515a81a-9aba-4752-ab2b-f7d68017f8f0 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:22.392242 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 559/1127] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:32:22 2022] GET /identity/v3/users/493489a2b3de475daa9d885dfb6f1bd7 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:22.397864 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf34ad-a497-42c7-bb7a-b4903ff46542 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:22.494159 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf34ad-a497-42c7-bb7a-b4903ff46542 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:22.526724 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82a9a2-1931-4757-a88d-76ec41734e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:22.527324 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82a9a2-1931-4757-a88d-76ec41734e2d None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:22.527603 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82a9a2-1931-4757-a88d-76ec41734e2d None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:22.528151 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82a9a2-1931-4757-a88d-76ec41734e2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:22.560747 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:22.593567 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d82a9a2-1931-4757-a88d-76ec41734e2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:22.595017 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d82a9a2-1931-4757-a88d-76ec41734e2d None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:22.687828 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 569/1128] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:32:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 23:32:22.693651 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3515a81a-9aba-4752-ab2b-f7d68017f8f0 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:22.790746 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3515a81a-9aba-4752-ab2b-f7d68017f8f0 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:22.822926 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1d0363-786e-4ba5-ac76-967289c669f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:22.823539 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d0363-786e-4ba5-ac76-967289c669f5 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:22.823819 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d0363-786e-4ba5-ac76-967289c669f5 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:22.824117 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d0363-786e-4ba5-ac76-967289c669f5 None admin] PATH_INFO: `/v3/projects/7d63c937687247a887936dbef97f25f0` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:22.825226 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1d0363-786e-4ba5-ac76-967289c669f5 None admin] RBAC: Authorizing `identity:get_project(project_id=7d63c937687247a887936dbef97f25f0)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:22.826654 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1d0363-786e-4ba5-ac76-967289c669f5 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:22.828280 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 560/1129] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:32:22 2022] GET /identity/v3/projects/7d63c937687247a887936dbef97f25f0 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:22.834129 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82a9a2-1931-4757-a88d-76ec41734e2d None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:22.895763 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:22.929786 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82a9a2-1931-4757-a88d-76ec41734e2d None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:22.962813 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6aabc6-054d-48fa-aa9e-a62fc7722288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:22.963350 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6aabc6-054d-48fa-aa9e-a62fc7722288 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:22.963629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6aabc6-054d-48fa-aa9e-a62fc7722288 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:22.963899 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6aabc6-054d-48fa-aa9e-a62fc7722288 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:22.964644 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6aabc6-054d-48fa-aa9e-a62fc7722288 None admin] RBAC: Adding query filter params (role.id=3428ff8e92274de6817b7e8ee9a56d13, scope.project.id=7d63c937687247a887936dbef97f25f0, user.id=493489a2b3de475daa9d885dfb6f1bd7) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:32:22.964951 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6aabc6-054d-48fa-aa9e-a62fc7722288 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:22.966367 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6aabc6-054d-48fa-aa9e-a62fc7722288 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:22.972025 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 570/1130] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:32:22 2022] GET /identity/v3/role_assignments?user.id=493489a2b3de475daa9d885dfb6f1bd7&scope.project.id=7d63c937687247a887936dbef97f25f0&role.id=3428ff8e92274de6817b7e8ee9a56d13 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:24.835489 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd6912-0482-4a00-a534-b6e11a3b6128 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:24.836553 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd6912-0482-4a00-a534-b6e11a3b6128 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:24.836553 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd6912-0482-4a00-a534-b6e11a3b6128 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:24.837898 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 561/1131] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:32:24.843189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8518d-4ee3-4e98-82c6-e73bbb9ce15f None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:24.843599 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8518d-4ee3-4e98-82c6-e73bbb9ce15f None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:24.843741 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8518d-4ee3-4e98-82c6-e73bbb9ce15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:24.861924 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-e5c8518d-4ee3-4e98-82c6-e73bbb9ce15f None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:32:24.863115 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-e5c8518d-4ee3-4e98-82c6-e73bbb9ce15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:32:24.927308 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 571/1132] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:32:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:32:25.191367 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd6912-0482-4a00-a534-b6e11a3b6128 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:25.207627 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:25.292094 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd6912-0482-4a00-a534-b6e11a3b6128 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:25.324313 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f282ab12-d55a-4fa4-9911-0872debd68e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:25.324905 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282ab12-d55a-4fa4-9911-0872debd68e2 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:25.325179 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282ab12-d55a-4fa4-9911-0872debd68e2 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:25.325444 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282ab12-d55a-4fa4-9911-0872debd68e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:25.326535 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f282ab12-d55a-4fa4-9911-0872debd68e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:25.327955 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f282ab12-d55a-4fa4-9911-0872debd68e2 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:25.329523 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 562/1133] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:32:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:25.335280 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c8518d-4ee3-4e98-82c6-e73bbb9ce15f None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:25.391188 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:25.430882 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c8518d-4ee3-4e98-82c6-e73bbb9ce15f None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:25.463698 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed5693-9e62-4dd3-be57-61de1f2a9728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:25.464336 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed5693-9e62-4dd3-be57-61de1f2a9728 None admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:25.464619 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed5693-9e62-4dd3-be57-61de1f2a9728 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:25.464893 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed5693-9e62-4dd3-be57-61de1f2a9728 None admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:25.465542 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fed5693-9e62-4dd3-be57-61de1f2a9728 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:25.466974 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fed5693-9e62-4dd3-be57-61de1f2a9728 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:25.493322 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 572/1134] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Aug 17 23:32:25 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:32:27.386090 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4524be1c-88ab-481e-aba7-be3e85be00f5 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:27.386651 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4524be1c-88ab-481e-aba7-be3e85be00f5 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:27.386651 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4524be1c-88ab-481e-aba7-be3e85be00f5 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:27.387998 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 563/1135] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:32:27.393197 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e70880-535a-4bf4-8de6-2c19e2e584d6 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:27.393613 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e70880-535a-4bf4-8de6-2c19e2e584d6 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:27.393758 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e70880-535a-4bf4-8de6-2c19e2e584d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:27.411589 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-e3e70880-535a-4bf4-8de6-2c19e2e584d6 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:32:27.412798 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-e3e70880-535a-4bf4-8de6-2c19e2e584d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:32:27.477574 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 573/1136] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:32:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:32:27.743054 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4524be1c-88ab-481e-aba7-be3e85be00f5 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:27.823422 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:27.842725 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4524be1c-88ab-481e-aba7-be3e85be00f5 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:27.875460 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02092d22-fdf1-4eca-8307-37d2aaf4e042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:27.876109 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02092d22-fdf1-4eca-8307-37d2aaf4e042 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:27.876389 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02092d22-fdf1-4eca-8307-37d2aaf4e042 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:27.876662 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02092d22-fdf1-4eca-8307-37d2aaf4e042 None admin] PATH_INFO: `/v3/domains/default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:27.877764 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02092d22-fdf1-4eca-8307-37d2aaf4e042 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:27.879195 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02092d22-fdf1-4eca-8307-37d2aaf4e042 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:27.880788 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 564/1137] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:32:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:27.886523 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e70880-535a-4bf4-8de6-2c19e2e584d6 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:27.928602 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:27.983500 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e70880-535a-4bf4-8de6-2c19e2e584d6 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:28.016505 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73564d2-8c96-4b68-aafb-1a03d5c69795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:28.017036 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73564d2-8c96-4b68-aafb-1a03d5c69795 None admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:28.017311 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73564d2-8c96-4b68-aafb-1a03d5c69795 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:28.017579 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73564d2-8c96-4b68-aafb-1a03d5c69795 None admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:28.018202 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73564d2-8c96-4b68-aafb-1a03d5c69795 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:28.019608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73564d2-8c96-4b68-aafb-1a03d5c69795 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:28.040073 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 574/1138] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Aug 17 23:32:27 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:32:29.957470 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9b48a-2186-44eb-a291-1a65da777716 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:29.957998 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9b48a-2186-44eb-a291-1a65da777716 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:29.958090 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9b48a-2186-44eb-a291-1a65da777716 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:29.959386 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 565/1139] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:32:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:32:29.965114 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c7c1e-9f8a-4e2e-9915-9c7f454f2677 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:29.965499 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c7c1e-9f8a-4e2e-9915-9c7f454f2677 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:29.965667 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c7c1e-9f8a-4e2e-9915-9c7f454f2677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:29.983737 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-0a1c7c1e-9f8a-4e2e-9915-9c7f454f2677 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:32:29.984939 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-0a1c7c1e-9f8a-4e2e-9915-9c7f454f2677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:32:30.047931 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 575/1140] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:32:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:32:30.316146 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b9b48a-2186-44eb-a291-1a65da777716 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:30.414979 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b9b48a-2186-44eb-a291-1a65da777716 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:30.449698 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e813944-956e-485c-bb40-0bb92ab52a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:30.450317 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e813944-956e-485c-bb40-0bb92ab52a2d None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:30.450595 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e813944-956e-485c-bb40-0bb92ab52a2d None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:30.450863 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e813944-956e-485c-bb40-0bb92ab52a2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:30.460812 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e813944-956e-485c-bb40-0bb92ab52a2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:32:30.465496 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-4e813944-956e-485c-bb40-0bb92ab52a2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:32:30.465496 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:32:30.465496 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:32:30.465496 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:32:30.465496 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:32:30.465496 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:32:30.465496 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:32:30.465496 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:32:30.465496 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:32:30.465496 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:32:30.465496 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:32:30.465496 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:32:30.465496 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:32:30.465496 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:32:30.465496 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:32:30.465496 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:32:30.466643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:32:30.466643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:32:30.466643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:32:30.466643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:32:30.466643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:32:30.466643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:32:30.466643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:32:30.466643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:32:30.466643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:32:30.466643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:32:30.466643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:32:30.466643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:32:30.466643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:32:30.466643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:32:30.466643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:32:30.466643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:32:30.467745 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:32:30.467745 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:32:30.467745 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:32:30.467745 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:32:30.467745 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:32:30.467745 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:32:30.467745 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:32:30.467745 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:32:30.467745 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:32:30.467745 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:32:30.467745 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:32:30.467745 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:32:30.467745 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 566/1141] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 17 23:32:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:32:30.472672 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c7c1e-9f8a-4e2e-9915-9c7f454f2677 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:30.500328 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:30.569836 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c7c1e-9f8a-4e2e-9915-9c7f454f2677 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:30.602168 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c15a083-cce9-4b4a-b4ac-736b73f777b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:30.602765 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c15a083-cce9-4b4a-b4ac-736b73f777b1 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:30.603045 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c15a083-cce9-4b4a-b4ac-736b73f777b1 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:30.603318 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c15a083-cce9-4b4a-b4ac-736b73f777b1 None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:30.604002 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c15a083-cce9-4b4a-b4ac-736b73f777b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:32:30.604304 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c15a083-cce9-4b4a-b4ac-736b73f777b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:30.605665 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c15a083-cce9-4b4a-b4ac-736b73f777b1 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:30.613381 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 576/1142] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:32:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:30.619691 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e813944-956e-485c-bb40-0bb92ab52a2d None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:30.632096 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:30.717049 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e813944-956e-485c-bb40-0bb92ab52a2d None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:30.749840 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956c9c1-65d8-4a74-8ecf-0999a68676bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:30.750451 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956c9c1-65d8-4a74-8ecf-0999a68676bc None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:30.750732 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956c9c1-65d8-4a74-8ecf-0999a68676bc None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:30.751006 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956c9c1-65d8-4a74-8ecf-0999a68676bc None admin] PATH_INFO: `/v3/users/0057dcb779024ba18af609c5ca5bfdec` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:30.763614 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e956c9c1-65d8-4a74-8ecf-0999a68676bc None admin] RBAC: Authorizing `identity:get_user(user_id=0057dcb779024ba18af609c5ca5bfdec)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:30.765082 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e956c9c1-65d8-4a74-8ecf-0999a68676bc None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:30.766663 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 567/1143] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:32:30 2022] GET /identity/v3/users/0057dcb779024ba18af609c5ca5bfdec => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:30.772568 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c15a083-cce9-4b4a-b4ac-736b73f777b1 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:30.864557 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:30.874119 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c15a083-cce9-4b4a-b4ac-736b73f777b1 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:30.906365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1cdf3b-f1e5-41c3-8dce-30c3b52e4c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:30.906955 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1cdf3b-f1e5-41c3-8dce-30c3b52e4c65 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:30.907234 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1cdf3b-f1e5-41c3-8dce-30c3b52e4c65 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:30.907503 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1cdf3b-f1e5-41c3-8dce-30c3b52e4c65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:30.971231 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1cdf3b-f1e5-41c3-8dce-30c3b52e4c65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:30.972697 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1cdf3b-f1e5-41c3-8dce-30c3b52e4c65 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:31.061181 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:31.071815 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 577/1144] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:32:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 23:32:31.077695 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956c9c1-65d8-4a74-8ecf-0999a68676bc None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:31.141023 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:31.173742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956c9c1-65d8-4a74-8ecf-0999a68676bc None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:31.206157 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5656ce-31c2-4031-8a6e-b84d8b4c2ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:31.206794 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5656ce-31c2-4031-8a6e-b84d8b4c2ff8 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:31.207073 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5656ce-31c2-4031-8a6e-b84d8b4c2ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:31.207343 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5656ce-31c2-4031-8a6e-b84d8b4c2ff8 None admin] PATH_INFO: `/v3/projects/e88b7eb56dc043c58bd7d86c0d094b7c` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:31.208461 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5656ce-31c2-4031-8a6e-b84d8b4c2ff8 None admin] RBAC: Authorizing `identity:get_project(project_id=e88b7eb56dc043c58bd7d86c0d094b7c)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:31.209884 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5656ce-31c2-4031-8a6e-b84d8b4c2ff8 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:31.212075 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 568/1145] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:32:31 2022] GET /identity/v3/projects/e88b7eb56dc043c58bd7d86c0d094b7c => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:31.218054 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1cdf3b-f1e5-41c3-8dce-30c3b52e4c65 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:31.312889 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1cdf3b-f1e5-41c3-8dce-30c3b52e4c65 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:31.345807 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e8535-ecca-45d0-a47f-794607aca497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:31.346395 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e8535-ecca-45d0-a47f-794607aca497 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:31.346674 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e8535-ecca-45d0-a47f-794607aca497 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:31.346943 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e8535-ecca-45d0-a47f-794607aca497 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:31.347650 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e8535-ecca-45d0-a47f-794607aca497 None admin] RBAC: Adding query filter params (role.id=33bd60d3b0e94dcbbfe97e87f91a8d78, scope.project.id=e88b7eb56dc043c58bd7d86c0d094b7c, user.id=0057dcb779024ba18af609c5ca5bfdec) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:32:31.347954 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e8535-ecca-45d0-a47f-794607aca497 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:31.349521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e8535-ecca-45d0-a47f-794607aca497 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:31.354706 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 578/1146] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:32:31 2022] GET /identity/v3/role_assignments?user.id=0057dcb779024ba18af609c5ca5bfdec&scope.project.id=e88b7eb56dc043c58bd7d86c0d094b7c&role.id=33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:33.182916 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0f40d-767d-49b3-8575-74b9d4710ac5 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:33.183467 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0f40d-767d-49b3-8575-74b9d4710ac5 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:33.183467 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0f40d-767d-49b3-8575-74b9d4710ac5 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:33.184808 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 569/1147] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:32:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:32:33.189971 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0f3c4-cb83-466b-af6f-e34e7fcbef9e None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:33.190333 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0f3c4-cb83-466b-af6f-e34e7fcbef9e None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:33.190522 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0f3c4-cb83-466b-af6f-e34e7fcbef9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:33.208716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-a7a0f3c4-cb83-466b-af6f-e34e7fcbef9e None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:32:33.209898 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-a7a0f3c4-cb83-466b-af6f-e34e7fcbef9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:32:33.257408 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:33.275398 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 579/1148] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:32:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:32:33.540925 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc0f40d-767d-49b3-8575-74b9d4710ac5 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:33.637703 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc0f40d-767d-49b3-8575-74b9d4710ac5 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:33.669553 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6eeb3-a556-4598-a7db-bcdcd30bfb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:33.670150 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6eeb3-a556-4598-a7db-bcdcd30bfb74 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:33.670422 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6eeb3-a556-4598-a7db-bcdcd30bfb74 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:33.670688 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6eeb3-a556-4598-a7db-bcdcd30bfb74 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:33.673252 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:33.681651 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f6eeb3-a556-4598-a7db-bcdcd30bfb74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:32:33.686266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-e8f6eeb3-a556-4598-a7db-bcdcd30bfb74 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:32:33.686266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:32:33.686266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:32:33.686266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:32:33.686266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:32:33.686266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:32:33.686266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:32:33.686266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:32:33.686266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:32:33.686266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:32:33.686266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:32:33.686266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:32:33.686266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:32:33.686266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:32:33.686266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:32:33.686266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:32:33.687381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:32:33.687381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:32:33.687381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:32:33.687381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:32:33.687381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:32:33.687381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:32:33.687381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:32:33.687381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:32:33.687381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:32:33.687381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:32:33.687381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:32:33.687381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:32:33.687381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:32:33.687381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:32:33.687381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:32:33.687381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:32:33.688430 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:32:33.688430 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:32:33.688430 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:32:33.688430 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:32:33.688430 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:32:33.688430 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:32:33.688430 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:32:33.688430 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:32:33.688430 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:32:33.688430 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:32:33.688430 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:32:33.688430 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:32:33.688430 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 570/1149] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 17 23:32:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:32:33.693191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a0f3c4-cb83-466b-af6f-e34e7fcbef9e None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:33.788647 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a0f3c4-cb83-466b-af6f-e34e7fcbef9e None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:33.821960 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32bcc10-a5cd-4cc2-b965-04ed8d4b13a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:33.822557 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32bcc10-a5cd-4cc2-b965-04ed8d4b13a7 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:33.822840 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32bcc10-a5cd-4cc2-b965-04ed8d4b13a7 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:33.823117 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32bcc10-a5cd-4cc2-b965-04ed8d4b13a7 None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:33.823784 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32bcc10-a5cd-4cc2-b965-04ed8d4b13a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:32:33.824118 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32bcc10-a5cd-4cc2-b965-04ed8d4b13a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:33.825520 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32bcc10-a5cd-4cc2-b965-04ed8d4b13a7 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:33.833285 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 580/1150] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:32:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:33.839266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6eeb3-a556-4598-a7db-bcdcd30bfb74 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:33.934583 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6eeb3-a556-4598-a7db-bcdcd30bfb74 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:33.967251 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b16e02-5339-47d6-b690-2161ec664b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:33.967846 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b16e02-5339-47d6-b690-2161ec664b7c None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:33.968148 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b16e02-5339-47d6-b690-2161ec664b7c None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:33.968416 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b16e02-5339-47d6-b690-2161ec664b7c None admin] PATH_INFO: `/v3/users/0057dcb779024ba18af609c5ca5bfdec` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:33.969509 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b16e02-5339-47d6-b690-2161ec664b7c None admin] RBAC: Authorizing `identity:get_user(user_id=0057dcb779024ba18af609c5ca5bfdec)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:33.970922 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b16e02-5339-47d6-b690-2161ec664b7c None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:33.972475 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 571/1151] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:32:33 2022] GET /identity/v3/users/0057dcb779024ba18af609c5ca5bfdec => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:32:33.978442 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32bcc10-a5cd-4cc2-b965-04ed8d4b13a7 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:34.023289 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:34.074733 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32bcc10-a5cd-4cc2-b965-04ed8d4b13a7 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:34.106395 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c98b446-6d70-41f5-a720-29c824ac2c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:34.106998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c98b446-6d70-41f5-a720-29c824ac2c52 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:34.107279 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c98b446-6d70-41f5-a720-29c824ac2c52 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:34.107544 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c98b446-6d70-41f5-a720-29c824ac2c52 None admin] PATH_INFO: `/v3/projects/e88b7eb56dc043c58bd7d86c0d094b7c` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:34.108639 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c98b446-6d70-41f5-a720-29c824ac2c52 None admin] RBAC: Authorizing `identity:get_project(project_id=e88b7eb56dc043c58bd7d86c0d094b7c)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:34.110041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c98b446-6d70-41f5-a720-29c824ac2c52 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:34.111610 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 581/1152] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:32:33 2022] GET /identity/v3/projects/e88b7eb56dc043c58bd7d86c0d094b7c => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:34.118037 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b16e02-5339-47d6-b690-2161ec664b7c None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:34.177980 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:34.214484 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b16e02-5339-47d6-b690-2161ec664b7c None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:34.247397 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8350ab8d-6766-42e6-a301-6a314a77ac72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:34.248068 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8350ab8d-6766-42e6-a301-6a314a77ac72 None admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:34.248347 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8350ab8d-6766-42e6-a301-6a314a77ac72 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:34.248618 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8350ab8d-6766-42e6-a301-6a314a77ac72 None admin] PATH_INFO: `/v3/projects/e88b7eb56dc043c58bd7d86c0d094b7c/users/0057dcb779024ba18af609c5ca5bfdec/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:34.250473 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8350ab8d-6766-42e6-a301-6a314a77ac72 None admin] RBAC: Authorizing `identity:create_grant(project_id=e88b7eb56dc043c58bd7d86c0d094b7c, user_id=0057dcb779024ba18af609c5ca5bfdec, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:34.251899 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8350ab8d-6766-42e6-a301-6a314a77ac72 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:34.264457 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 572/1153] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 17 23:32:34 2022] PUT /identity/v3/projects/e88b7eb56dc043c58bd7d86c0d094b7c/users/0057dcb779024ba18af609c5ca5bfdec/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:32:36.111032 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a008d-7d66-4271-9816-caedbeabd7c8 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:36.111556 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a008d-7d66-4271-9816-caedbeabd7c8 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:36.111556 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a008d-7d66-4271-9816-caedbeabd7c8 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:36.112923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 582/1154] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:32:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:32:36.118080 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f585f71-56c2-4b4a-b404-7cec85401407 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:36.118475 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f585f71-56c2-4b4a-b404-7cec85401407 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:36.118623 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f585f71-56c2-4b4a-b404-7cec85401407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:36.135943 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-4f585f71-56c2-4b4a-b404-7cec85401407 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:32:36.137125 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-4f585f71-56c2-4b4a-b404-7cec85401407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:32:36.201243 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 573/1155] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:32:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:32:36.467745 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a008d-7d66-4271-9816-caedbeabd7c8 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:36.565538 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a008d-7d66-4271-9816-caedbeabd7c8 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:36.578324 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:36.598110 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659ab38-37c7-4806-8c91-b44e2d8405cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:36.598728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659ab38-37c7-4806-8c91-b44e2d8405cc None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:36.599005 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659ab38-37c7-4806-8c91-b44e2d8405cc None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:36.599275 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659ab38-37c7-4806-8c91-b44e2d8405cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:36.609908 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3659ab38-37c7-4806-8c91-b44e2d8405cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:32:36.614579 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-3659ab38-37c7-4806-8c91-b44e2d8405cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:32:36.614579 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:32:36.614579 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:32:36.614579 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:32:36.614579 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:32:36.614579 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:32:36.614579 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:32:36.614579 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:32:36.614579 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:32:36.614579 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:32:36.614579 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:32:36.614579 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:32:36.614579 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:32:36.614579 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:32:36.614579 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:32:36.614579 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:32:36.615670 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:32:36.615670 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:32:36.615670 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:32:36.615670 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:32:36.615670 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:32:36.615670 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:32:36.615670 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:32:36.615670 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:32:36.615670 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:32:36.615670 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:32:36.615670 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:32:36.615670 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:32:36.615670 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:32:36.615670 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:32:36.615670 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:32:36.615670 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:32:36.616586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:32:36.616586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:32:36.616586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:32:36.616586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:32:36.616586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:32:36.616586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:32:36.616586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:32:36.616586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:32:36.616586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:32:36.616586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:32:36.616586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:32:36.616586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:32:36.616586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 583/1156] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 17 23:32:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:32:36.622200 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f585f71-56c2-4b4a-b404-7cec85401407 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:36.675552 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:36.720173 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f585f71-56c2-4b4a-b404-7cec85401407 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:36.752858 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f79dcf-e7d0-444e-8c51-e5eaa556a00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:36.753455 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f79dcf-e7d0-444e-8c51-e5eaa556a00c None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:36.753733 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f79dcf-e7d0-444e-8c51-e5eaa556a00c None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:36.754005 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f79dcf-e7d0-444e-8c51-e5eaa556a00c None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:36.754663 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f79dcf-e7d0-444e-8c51-e5eaa556a00c None admin] RBAC: Adding query filter params (name=admin) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:32:36.754965 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f79dcf-e7d0-444e-8c51-e5eaa556a00c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:36.756367 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f79dcf-e7d0-444e-8c51-e5eaa556a00c None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:36.764266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 574/1157] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:32:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:36.770084 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659ab38-37c7-4806-8c91-b44e2d8405cc None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:36.865752 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659ab38-37c7-4806-8c91-b44e2d8405cc None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:36.896867 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b58d8f-9729-4340-a1de-4c632ce57a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:36.897474 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b58d8f-9729-4340-a1de-4c632ce57a64 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:36.897749 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b58d8f-9729-4340-a1de-4c632ce57a64 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:36.898526 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b58d8f-9729-4340-a1de-4c632ce57a64 None admin] PATH_INFO: `/v3/users/0057dcb779024ba18af609c5ca5bfdec` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:36.899617 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b58d8f-9729-4340-a1de-4c632ce57a64 None admin] RBAC: Authorizing `identity:get_user(user_id=0057dcb779024ba18af609c5ca5bfdec)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:36.901058 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b58d8f-9729-4340-a1de-4c632ce57a64 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:36.902593 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 584/1158] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:32:36 2022] GET /identity/v3/users/0057dcb779024ba18af609c5ca5bfdec => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:36.908655 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f79dcf-e7d0-444e-8c51-e5eaa556a00c None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:37.005365 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f79dcf-e7d0-444e-8c51-e5eaa556a00c None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:37.020969 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:37.038978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b796510-7619-4c0d-893a-5e793bd42eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:37.039570 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b796510-7619-4c0d-893a-5e793bd42eb7 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:37.039848 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b796510-7619-4c0d-893a-5e793bd42eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:37.040162 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b796510-7619-4c0d-893a-5e793bd42eb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:37.105034 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b796510-7619-4c0d-893a-5e793bd42eb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:37.106477 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b796510-7619-4c0d-893a-5e793bd42eb7 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:37.200315 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 575/1159] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:32:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 23:32:37.206165 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b58d8f-9729-4340-a1de-4c632ce57a64 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:37.247507 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:37.302850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b58d8f-9729-4340-a1de-4c632ce57a64 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:37.334386 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd071f2-53db-4a1f-8844-b58ef9e580d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:37.334986 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd071f2-53db-4a1f-8844-b58ef9e580d1 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:37.335261 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd071f2-53db-4a1f-8844-b58ef9e580d1 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:37.336151 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd071f2-53db-4a1f-8844-b58ef9e580d1 None admin] PATH_INFO: `/v3/projects/e88b7eb56dc043c58bd7d86c0d094b7c` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:37.337939 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd071f2-53db-4a1f-8844-b58ef9e580d1 None admin] RBAC: Authorizing `identity:get_project(project_id=e88b7eb56dc043c58bd7d86c0d094b7c)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:37.338663 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd071f2-53db-4a1f-8844-b58ef9e580d1 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:37.340277 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 585/1160] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:32:37 2022] GET /identity/v3/projects/e88b7eb56dc043c58bd7d86c0d094b7c => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:37.346265 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b796510-7619-4c0d-893a-5e793bd42eb7 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:37.362969 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:37.442820 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b796510-7619-4c0d-893a-5e793bd42eb7 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:37.476206 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f609f-7593-4c25-804d-f7e72d0df424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:37.476839 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f609f-7593-4c25-804d-f7e72d0df424 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:37.477122 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f609f-7593-4c25-804d-f7e72d0df424 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:37.477412 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f609f-7593-4c25-804d-f7e72d0df424 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:37.478134 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425f609f-7593-4c25-804d-f7e72d0df424 None admin] RBAC: Adding query filter params (role.id=33bd60d3b0e94dcbbfe97e87f91a8d78, scope.project.id=e88b7eb56dc043c58bd7d86c0d094b7c, user.id=0057dcb779024ba18af609c5ca5bfdec) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:32:37.478440 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425f609f-7593-4c25-804d-f7e72d0df424 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:37.479873 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425f609f-7593-4c25-804d-f7e72d0df424 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:37.485483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 576/1161] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:32:37 2022] GET /identity/v3/role_assignments?user.id=0057dcb779024ba18af609c5ca5bfdec&scope.project.id=e88b7eb56dc043c58bd7d86c0d094b7c&role.id=33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:39.357325 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73919010-68d3-4c65-95e4-abc010149670 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:39.357852 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73919010-68d3-4c65-95e4-abc010149670 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:39.357852 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73919010-68d3-4c65-95e4-abc010149670 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:39.359210 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 586/1162] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:32:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:32:39.364500 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0d66f-6cee-43e5-ba44-3ba40aaa1277 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:39.364906 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0d66f-6cee-43e5-ba44-3ba40aaa1277 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:39.365068 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0d66f-6cee-43e5-ba44-3ba40aaa1277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:39.383123 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-f2c0d66f-6cee-43e5-ba44-3ba40aaa1277 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:32:39.384336 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-f2c0d66f-6cee-43e5-ba44-3ba40aaa1277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:32:39.448822 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 577/1163] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:32:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:32:39.712952 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73919010-68d3-4c65-95e4-abc010149670 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:39.809936 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73919010-68d3-4c65-95e4-abc010149670 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:39.820148 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:39.843385 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597c537-5bdc-40ea-bf21-9606fd38c5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:39.844012 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597c537-5bdc-40ea-bf21-9606fd38c5c0 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:39.844295 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597c537-5bdc-40ea-bf21-9606fd38c5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:39.844572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597c537-5bdc-40ea-bf21-9606fd38c5c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:39.908622 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d597c537-5bdc-40ea-bf21-9606fd38c5c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:39.910066 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d597c537-5bdc-40ea-bf21-9606fd38c5c0 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:40.003288 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 587/1164] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:32:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 23:32:40.009720 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c0d66f-6cee-43e5-ba44-3ba40aaa1277 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:40.020839 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:40.106213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c0d66f-6cee-43e5-ba44-3ba40aaa1277 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:40.138607 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3a0817-baf4-42d2-85d3-2fceef332e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:40.139209 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a0817-baf4-42d2-85d3-2fceef332e56 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:40.139482 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a0817-baf4-42d2-85d3-2fceef332e56 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:40.139746 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a0817-baf4-42d2-85d3-2fceef332e56 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:40.146126 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3a0817-baf4-42d2-85d3-2fceef332e56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:40.147623 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3a0817-baf4-42d2-85d3-2fceef332e56 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:40.159290 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-2b3a0817-baf4-42d2-85d3-2fceef332e56 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 17 23:32:40.159290 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:32:40.159290 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 17 23:32:40.159290 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 17 23:32:40.159290 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 23:32:40.159290 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 23:32:40.159290 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 23:32:40.159290 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 23:32:40.159290 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 17 23:32:40.159290 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:32:40.159290 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 23:32:40.159290 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:32:40.159290 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:32:40.159290 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:32:40.159290 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:32:40.159290 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:32:40.159290 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:32:40.160522 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:32:40.160522 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:32:40.160522 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:32:40.160522 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:32:40.160522 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:32:40.160522 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:32:40.160522 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:32:40.160522 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:32:40.160522 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:32:40.160522 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:32:40.160522 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:32:40.160522 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:32:40.160522 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:32:40.160522 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:32:40.160522 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:32:40.160522 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:32:40.161567 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:32:40.161567 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:32:40.161567 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:32:40.161567 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:32:40.161567 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:32:40.161567 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:32:40.161567 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:32:40.161567 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:32:40.161567 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:32:40.161567 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:32:40.161567 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 17 23:32:40.161567 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:32:40.161567 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 17 23:32:40.161567 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:32:40.161567 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 578/1165] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Aug 17 23:32:40 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:32:40.166270 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597c537-5bdc-40ea-bf21-9606fd38c5c0 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:40.177350 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:40.260387 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597c537-5bdc-40ea-bf21-9606fd38c5c0 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:40.292597 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c796f7-bbf6-47ab-b8de-04f30bc6e46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:40.293184 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c796f7-bbf6-47ab-b8de-04f30bc6e46a None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:40.293460 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c796f7-bbf6-47ab-b8de-04f30bc6e46a None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:40.293729 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c796f7-bbf6-47ab-b8de-04f30bc6e46a None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:40.294354 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c796f7-bbf6-47ab-b8de-04f30bc6e46a None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:32:40.294650 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c796f7-bbf6-47ab-b8de-04f30bc6e46a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:40.296021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c796f7-bbf6-47ab-b8de-04f30bc6e46a None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:40.302790 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 588/1166] 10.222.0.26 () {56 vars in 1115 bytes} [Wed Aug 17 23:32:40 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:42.155403 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb188ea-ff56-44fa-90a1-5d2b0849b855 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:42.155988 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb188ea-ff56-44fa-90a1-5d2b0849b855 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:42.155988 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb188ea-ff56-44fa-90a1-5d2b0849b855 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:42.157328 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 579/1167] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:32:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:32:42.162434 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940d240-3847-496e-aca4-744158551f06 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:42.162836 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940d240-3847-496e-aca4-744158551f06 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:42.162979 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940d240-3847-496e-aca4-744158551f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:42.180883 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-e940d240-3847-496e-aca4-744158551f06 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:32:42.182059 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-e940d240-3847-496e-aca4-744158551f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:32:42.244859 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 589/1168] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:32:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:32:42.507922 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb188ea-ff56-44fa-90a1-5d2b0849b855 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:42.576276 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:42.607709 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb188ea-ff56-44fa-90a1-5d2b0849b855 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:42.641103 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da01d1c-95c5-486e-a4e5-131e235fd0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:42.641690 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da01d1c-95c5-486e-a4e5-131e235fd0cb None admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:42.641969 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da01d1c-95c5-486e-a4e5-131e235fd0cb None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:42.642236 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da01d1c-95c5-486e-a4e5-131e235fd0cb None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:42.642896 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da01d1c-95c5-486e-a4e5-131e235fd0cb None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:42.644299 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da01d1c-95c5-486e-a4e5-131e235fd0cb None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:42.659395 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 580/1169] 10.222.0.26 () {60 vars in 1138 bytes} [Wed Aug 17 23:32:42 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:32:44.556106 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f49e6a5-c49e-42e8-8db6-32ffccf55113 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:44.556594 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f49e6a5-c49e-42e8-8db6-32ffccf55113 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:44.556687 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f49e6a5-c49e-42e8-8db6-32ffccf55113 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:44.557974 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 590/1170] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:32:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:32:44.562986 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a01ba7-31ab-41bb-9a37-637c19d4be55 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:44.563394 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a01ba7-31ab-41bb-9a37-637c19d4be55 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:44.563535 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a01ba7-31ab-41bb-9a37-637c19d4be55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:44.581591 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-85a01ba7-31ab-41bb-9a37-637c19d4be55 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:32:44.582762 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-85a01ba7-31ab-41bb-9a37-637c19d4be55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:32:44.646624 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 581/1171] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:32:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:32:44.910292 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f49e6a5-c49e-42e8-8db6-32ffccf55113 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:44.914607 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:45.007037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f49e6a5-c49e-42e8-8db6-32ffccf55113 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:45.040071 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab787b-2408-4519-9455-467a81a1744b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:45.040681 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab787b-2408-4519-9455-467a81a1744b None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:45.041017 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab787b-2408-4519-9455-467a81a1744b None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:45.041226 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab787b-2408-4519-9455-467a81a1744b None admin] PATH_INFO: `/v3/domains/ad08a1f0bb3b4053b675739bdb2457a6` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:45.051085 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aab787b-2408-4519-9455-467a81a1744b None admin] RBAC: Authorizing `identity:get_domain(domain_id=ad08a1f0bb3b4053b675739bdb2457a6)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:45.052554 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aab787b-2408-4519-9455-467a81a1744b None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:45.054121 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 591/1172] 10.222.0.26 () {56 vars in 1183 bytes} [Wed Aug 17 23:32:44 2022] GET /identity/v3/domains/ad08a1f0bb3b4053b675739bdb2457a6 => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:45.060058 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a01ba7-31ab-41bb-9a37-637c19d4be55 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:45.121606 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:45.161915 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a01ba7-31ab-41bb-9a37-637c19d4be55 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:45.194456 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d11bc5e-56c4-4c34-ad7b-7f15534d986c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:45.195066 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d11bc5e-56c4-4c34-ad7b-7f15534d986c None admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:45.195341 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d11bc5e-56c4-4c34-ad7b-7f15534d986c None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:45.195611 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d11bc5e-56c4-4c34-ad7b-7f15534d986c None admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:45.196270 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d11bc5e-56c4-4c34-ad7b-7f15534d986c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:45.197698 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d11bc5e-56c4-4c34-ad7b-7f15534d986c None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:45.222653 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 582/1173] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Aug 17 23:32:45 2022] POST /identity/v3/projects => generated 361 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:32:47.122016 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b0b02-6c1e-483d-9258-75ab720d0c93 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:47.122642 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b0b02-6c1e-483d-9258-75ab720d0c93 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:47.122642 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b0b02-6c1e-483d-9258-75ab720d0c93 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:47.123884 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 592/1174] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:32:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:32:47.129191 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe954f-7167-42ee-9451-8277eccdb354 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:47.129597 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe954f-7167-42ee-9451-8277eccdb354 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:47.129746 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe954f-7167-42ee-9451-8277eccdb354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:47.148507 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-4afe954f-7167-42ee-9451-8277eccdb354 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:32:47.149683 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-4afe954f-7167-42ee-9451-8277eccdb354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:32:47.213372 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 583/1175] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:32:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:32:47.485353 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b0b02-6c1e-483d-9258-75ab720d0c93 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:47.547387 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:47.585035 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b0b02-6c1e-483d-9258-75ab720d0c93 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:47.616923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec68e356-ca95-45af-9699-59b2f2265332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:47.617543 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec68e356-ca95-45af-9699-59b2f2265332 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:47.617825 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec68e356-ca95-45af-9699-59b2f2265332 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:47.618098 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec68e356-ca95-45af-9699-59b2f2265332 None admin] PATH_INFO: `/v3/domains/ad08a1f0bb3b4053b675739bdb2457a6` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:47.619202 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec68e356-ca95-45af-9699-59b2f2265332 None admin] RBAC: Authorizing `identity:get_domain(domain_id=ad08a1f0bb3b4053b675739bdb2457a6)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:47.620656 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec68e356-ca95-45af-9699-59b2f2265332 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:47.622200 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 593/1176] 10.222.0.26 () {56 vars in 1183 bytes} [Wed Aug 17 23:32:47 2022] GET /identity/v3/domains/ad08a1f0bb3b4053b675739bdb2457a6 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:47.627886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe954f-7167-42ee-9451-8277eccdb354 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:47.670141 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:47.725885 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe954f-7167-42ee-9451-8277eccdb354 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:47.758268 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a5b5d-b05f-4428-a59d-38435b63249a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:47.758852 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a5b5d-b05f-4428-a59d-38435b63249a None admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:47.759126 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a5b5d-b05f-4428-a59d-38435b63249a None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:47.759390 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a5b5d-b05f-4428-a59d-38435b63249a None admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:47.760053 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2a5b5d-b05f-4428-a59d-38435b63249a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:47.761491 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2a5b5d-b05f-4428-a59d-38435b63249a None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:47.782849 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 584/1177] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Aug 17 23:32:47 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:32:49.693320 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d936a6b-bf4d-4b82-8440-2ec7a669e8b7 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:49.693854 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d936a6b-bf4d-4b82-8440-2ec7a669e8b7 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:49.693854 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d936a6b-bf4d-4b82-8440-2ec7a669e8b7 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:49.695184 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 594/1178] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:32:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:32:49.700368 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d94301-34c6-4871-8765-8a773d350ab7 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:49.700783 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d94301-34c6-4871-8765-8a773d350ab7 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:49.700913 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d94301-34c6-4871-8765-8a773d350ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:49.719560 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-78d94301-34c6-4871-8765-8a773d350ab7 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:32:49.720768 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-78d94301-34c6-4871-8765-8a773d350ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:32:49.785314 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 585/1179] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:32:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:32:50.053488 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d936a6b-bf4d-4b82-8440-2ec7a669e8b7 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:50.151409 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d936a6b-bf4d-4b82-8440-2ec7a669e8b7 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:50.179215 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:50.184372 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40ee71-c433-42a9-8ec9-26b2b7e8ae23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:50.184989 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40ee71-c433-42a9-8ec9-26b2b7e8ae23 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:50.185270 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40ee71-c433-42a9-8ec9-26b2b7e8ae23 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:50.185545 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40ee71-c433-42a9-8ec9-26b2b7e8ae23 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:50.196387 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee40ee71-c433-42a9-8ec9-26b2b7e8ae23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:32:50.201075 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-ee40ee71-c433-42a9-8ec9-26b2b7e8ae23 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:32:50.201075 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:32:50.201075 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:32:50.201075 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:32:50.201075 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:32:50.201075 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:32:50.201075 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:32:50.201075 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:32:50.201075 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:32:50.201075 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:32:50.201075 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:32:50.201075 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:32:50.201075 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:32:50.201075 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:32:50.201075 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:32:50.201075 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:32:50.202217 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:32:50.202217 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:32:50.202217 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:32:50.202217 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:32:50.202217 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:32:50.202217 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:32:50.202217 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:32:50.202217 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:32:50.202217 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:32:50.202217 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:32:50.202217 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:32:50.202217 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:32:50.202217 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:32:50.202217 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:32:50.202217 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:32:50.202217 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:32:50.203130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:32:50.203130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:32:50.203130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:32:50.203130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:32:50.203130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:32:50.203130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:32:50.203130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:32:50.203130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:32:50.203130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:32:50.203130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:32:50.203130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:32:50.203130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:32:50.203130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 595/1180] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 17 23:32:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:32:50.208121 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d94301-34c6-4871-8765-8a773d350ab7 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:50.234229 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:50.303129 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d94301-34c6-4871-8765-8a773d350ab7 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:50.336172 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe8c9b1-b510-4f1a-acd1-99d3130bf938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:50.336711 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8c9b1-b510-4f1a-acd1-99d3130bf938 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:50.336990 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8c9b1-b510-4f1a-acd1-99d3130bf938 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:50.337259 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8c9b1-b510-4f1a-acd1-99d3130bf938 None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:50.337918 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe8c9b1-b510-4f1a-acd1-99d3130bf938 None admin] RBAC: Adding query filter params (name=admin) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:32:50.338220 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe8c9b1-b510-4f1a-acd1-99d3130bf938 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:50.339579 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe8c9b1-b510-4f1a-acd1-99d3130bf938 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:50.347293 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 586/1181] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:32:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:50.352949 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40ee71-c433-42a9-8ec9-26b2b7e8ae23 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:50.448551 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40ee71-c433-42a9-8ec9-26b2b7e8ae23 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:50.480801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e726d-e98f-4b36-af72-2adcaa2de8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:50.481410 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e726d-e98f-4b36-af72-2adcaa2de8f5 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:50.481686 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e726d-e98f-4b36-af72-2adcaa2de8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:50.481952 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e726d-e98f-4b36-af72-2adcaa2de8f5 None admin] PATH_INFO: `/v3/users/e88ca1ee86e24644b491d1eb34855b77` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:50.495099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935e726d-e98f-4b36-af72-2adcaa2de8f5 None admin] RBAC: Authorizing `identity:get_user(user_id=e88ca1ee86e24644b491d1eb34855b77)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:50.496613 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935e726d-e98f-4b36-af72-2adcaa2de8f5 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:50.498255 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 596/1182] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:32:50 2022] GET /identity/v3/users/e88ca1ee86e24644b491d1eb34855b77 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:50.504098 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe8c9b1-b510-4f1a-acd1-99d3130bf938 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:50.590952 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:50.600077 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe8c9b1-b510-4f1a-acd1-99d3130bf938 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:50.632729 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0925bc9f-41a1-4fe4-a9b0-7d3fb24f0bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:50.633315 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925bc9f-41a1-4fe4-a9b0-7d3fb24f0bc9 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:50.633593 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925bc9f-41a1-4fe4-a9b0-7d3fb24f0bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:50.633864 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925bc9f-41a1-4fe4-a9b0-7d3fb24f0bc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:50.698643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0925bc9f-41a1-4fe4-a9b0-7d3fb24f0bc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:50.700101 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0925bc9f-41a1-4fe4-a9b0-7d3fb24f0bc9 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:50.784570 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:50.795216 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 587/1183] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:32:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 23:32:50.801097 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e726d-e98f-4b36-af72-2adcaa2de8f5 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:50.848685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:50.896403 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e726d-e98f-4b36-af72-2adcaa2de8f5 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:50.929545 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce1e385-4a0f-472f-8468-e7b282bc50ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:50.930158 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1e385-4a0f-472f-8468-e7b282bc50ec None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:50.930440 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1e385-4a0f-472f-8468-e7b282bc50ec None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:50.930716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1e385-4a0f-472f-8468-e7b282bc50ec None admin] PATH_INFO: `/v3/projects/eb1683a1594744bbb71ad7cb7f3561c2` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:50.931822 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce1e385-4a0f-472f-8468-e7b282bc50ec None admin] RBAC: Authorizing `identity:get_project(project_id=eb1683a1594744bbb71ad7cb7f3561c2)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:50.933276 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce1e385-4a0f-472f-8468-e7b282bc50ec None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:50.934871 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 597/1184] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:32:50 2022] GET /identity/v3/projects/eb1683a1594744bbb71ad7cb7f3561c2 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:50.941184 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0925bc9f-41a1-4fe4-a9b0-7d3fb24f0bc9 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:51.035673 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0925bc9f-41a1-4fe4-a9b0-7d3fb24f0bc9 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:51.068085 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5beb5a-ab96-4003-a067-1c3758cc5353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:51.068662 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5beb5a-ab96-4003-a067-1c3758cc5353 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:51.068934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5beb5a-ab96-4003-a067-1c3758cc5353 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:51.069197 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5beb5a-ab96-4003-a067-1c3758cc5353 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:51.069930 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5beb5a-ab96-4003-a067-1c3758cc5353 None admin] RBAC: Adding query filter params (role.id=33bd60d3b0e94dcbbfe97e87f91a8d78, scope.project.id=eb1683a1594744bbb71ad7cb7f3561c2, user.id=e88ca1ee86e24644b491d1eb34855b77) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:32:51.070232 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5beb5a-ab96-4003-a067-1c3758cc5353 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:51.071631 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5beb5a-ab96-4003-a067-1c3758cc5353 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:51.076803 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 588/1185] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:32:50 2022] GET /identity/v3/role_assignments?user.id=e88ca1ee86e24644b491d1eb34855b77&scope.project.id=eb1683a1594744bbb71ad7cb7f3561c2&role.id=33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:32:52.896965 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d2c53-1623-4a2d-bed5-9e3b50538045 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:52.898002 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d2c53-1623-4a2d-bed5-9e3b50538045 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:52.898124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d2c53-1623-4a2d-bed5-9e3b50538045 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:52.899374 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 598/1186] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:32:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:32:52.904671 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75f2a7-756f-4868-8e2e-2bd1c62d828f None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:52.905106 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75f2a7-756f-4868-8e2e-2bd1c62d828f None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:52.905240 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75f2a7-756f-4868-8e2e-2bd1c62d828f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:52.922749 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-ad75f2a7-756f-4868-8e2e-2bd1c62d828f None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:32:52.923928 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-ad75f2a7-756f-4868-8e2e-2bd1c62d828f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:32:52.971783 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:52.988883 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 589/1187] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:32:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:32:53.258466 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d2c53-1623-4a2d-bed5-9e3b50538045 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:53.356070 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d2c53-1623-4a2d-bed5-9e3b50538045 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:53.372849 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:53.389581 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcbe7cb-98fe-4323-a888-f615d34f5bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:53.390194 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbe7cb-98fe-4323-a888-f615d34f5bd5 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:53.390475 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbe7cb-98fe-4323-a888-f615d34f5bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:53.390747 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbe7cb-98fe-4323-a888-f615d34f5bd5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:53.400496 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcbe7cb-98fe-4323-a888-f615d34f5bd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:32:53.405454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-adcbe7cb-98fe-4323-a888-f615d34f5bd5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:32:53.405454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:32:53.405454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:32:53.405454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:32:53.405454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:32:53.405454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:32:53.405454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:32:53.405454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:32:53.405454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:32:53.405454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:32:53.405454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:32:53.405454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:32:53.405454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:32:53.405454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:32:53.405454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:32:53.405454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:32:53.406542 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:32:53.406542 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:32:53.406542 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:32:53.406542 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:32:53.406542 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:32:53.406542 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:32:53.406542 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:32:53.406542 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:32:53.406542 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:32:53.406542 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:32:53.406542 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:32:53.406542 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:32:53.406542 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:32:53.406542 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:32:53.406542 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:32:53.406542 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:32:53.407594 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:32:53.407594 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:32:53.407594 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:32:53.407594 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:32:53.407594 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:32:53.407594 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:32:53.407594 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:32:53.407594 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:32:53.407594 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:32:53.407594 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:32:53.407594 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:32:53.407594 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:32:53.407594 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 599/1188] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 17 23:32:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:32:53.412320 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad75f2a7-756f-4868-8e2e-2bd1c62d828f None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:53.510196 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad75f2a7-756f-4868-8e2e-2bd1c62d828f None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:53.543831 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d944646-ab72-475c-88ff-bfc93f2adccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:53.544455 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d944646-ab72-475c-88ff-bfc93f2adccf None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:53.544731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d944646-ab72-475c-88ff-bfc93f2adccf None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:53.544998 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d944646-ab72-475c-88ff-bfc93f2adccf None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:53.545651 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d944646-ab72-475c-88ff-bfc93f2adccf None admin] RBAC: Adding query filter params (name=admin) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:32:53.545949 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d944646-ab72-475c-88ff-bfc93f2adccf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:53.547299 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d944646-ab72-475c-88ff-bfc93f2adccf None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:53.555667 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 590/1189] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:32:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:53.561567 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcbe7cb-98fe-4323-a888-f615d34f5bd5 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:53.656514 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcbe7cb-98fe-4323-a888-f615d34f5bd5 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:53.688977 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6fc2dd-fbc9-4216-81fd-cfb0b4c57580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:53.689580 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fc2dd-fbc9-4216-81fd-cfb0b4c57580 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:53.689853 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fc2dd-fbc9-4216-81fd-cfb0b4c57580 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:53.690121 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fc2dd-fbc9-4216-81fd-cfb0b4c57580 None admin] PATH_INFO: `/v3/users/e88ca1ee86e24644b491d1eb34855b77` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:53.691352 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6fc2dd-fbc9-4216-81fd-cfb0b4c57580 None admin] RBAC: Authorizing `identity:get_user(user_id=e88ca1ee86e24644b491d1eb34855b77)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:53.693461 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6fc2dd-fbc9-4216-81fd-cfb0b4c57580 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:53.696321 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 600/1190] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:32:53 2022] GET /identity/v3/users/e88ca1ee86e24644b491d1eb34855b77 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:53.702031 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d944646-ab72-475c-88ff-bfc93f2adccf None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:53.747931 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:53.799726 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d944646-ab72-475c-88ff-bfc93f2adccf None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:53.832352 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154797e4-7ac7-420f-948b-bb53469e50e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:53.832967 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154797e4-7ac7-420f-948b-bb53469e50e8 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:53.833250 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154797e4-7ac7-420f-948b-bb53469e50e8 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:53.833526 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154797e4-7ac7-420f-948b-bb53469e50e8 None admin] PATH_INFO: `/v3/projects/eb1683a1594744bbb71ad7cb7f3561c2` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:53.834653 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154797e4-7ac7-420f-948b-bb53469e50e8 None admin] RBAC: Authorizing `identity:get_project(project_id=eb1683a1594744bbb71ad7cb7f3561c2)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:53.836177 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154797e4-7ac7-420f-948b-bb53469e50e8 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:53.837824 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 591/1191] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:32:53 2022] GET /identity/v3/projects/eb1683a1594744bbb71ad7cb7f3561c2 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:53.843648 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6fc2dd-fbc9-4216-81fd-cfb0b4c57580 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:53.889114 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:53.941199 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6fc2dd-fbc9-4216-81fd-cfb0b4c57580 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:53.973559 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b81355-c526-41a1-9676-306d8c0a0e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:53.974206 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b81355-c526-41a1-9676-306d8c0a0e49 None admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:53.974485 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b81355-c526-41a1-9676-306d8c0a0e49 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:53.974759 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b81355-c526-41a1-9676-306d8c0a0e49 None admin] PATH_INFO: `/v3/projects/eb1683a1594744bbb71ad7cb7f3561c2/users/e88ca1ee86e24644b491d1eb34855b77/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:53.977238 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b81355-c526-41a1-9676-306d8c0a0e49 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb1683a1594744bbb71ad7cb7f3561c2, user_id=e88ca1ee86e24644b491d1eb34855b77, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:53.978670 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b81355-c526-41a1-9676-306d8c0a0e49 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:53.990591 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 601/1192] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 17 23:32:53 2022] PUT /identity/v3/projects/eb1683a1594744bbb71ad7cb7f3561c2/users/e88ca1ee86e24644b491d1eb34855b77/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:32:55.831598 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72566bcb-4ca2-4fd5-852b-c43bd6f024b1 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:55.832152 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72566bcb-4ca2-4fd5-852b-c43bd6f024b1 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:55.832152 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72566bcb-4ca2-4fd5-852b-c43bd6f024b1 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:55.833491 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 592/1193] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:32:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:32:55.839159 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0fcffb-53a9-4ad6-a8e0-554267679394 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:55.839524 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0fcffb-53a9-4ad6-a8e0-554267679394 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:55.839707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0fcffb-53a9-4ad6-a8e0-554267679394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:55.857818 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-2c0fcffb-53a9-4ad6-a8e0-554267679394 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:32:55.859011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-2c0fcffb-53a9-4ad6-a8e0-554267679394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:32:55.923283 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 602/1194] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:32:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:32:56.190642 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72566bcb-4ca2-4fd5-852b-c43bd6f024b1 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:56.290884 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72566bcb-4ca2-4fd5-852b-c43bd6f024b1 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:56.304775 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:56.325181 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd46f19b-83f0-472a-8cd5-42133ef6bc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:56.325794 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46f19b-83f0-472a-8cd5-42133ef6bc95 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:56.326075 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46f19b-83f0-472a-8cd5-42133ef6bc95 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:56.326348 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46f19b-83f0-472a-8cd5-42133ef6bc95 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:56.336157 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd46f19b-83f0-472a-8cd5-42133ef6bc95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:32:56.340831 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-fd46f19b-83f0-472a-8cd5-42133ef6bc95 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:32:56.340831 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:32:56.340831 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:32:56.340831 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:32:56.340831 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:32:56.340831 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:32:56.340831 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:32:56.340831 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:32:56.340831 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:32:56.340831 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:32:56.340831 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:32:56.340831 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:32:56.340831 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:32:56.340831 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:32:56.340831 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:32:56.340831 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:32:56.341979 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:32:56.341979 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:32:56.341979 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:32:56.341979 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:32:56.341979 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:32:56.341979 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:32:56.341979 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:32:56.341979 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:32:56.341979 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:32:56.341979 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:32:56.341979 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:32:56.341979 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:32:56.341979 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:32:56.341979 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:32:56.341979 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:32:56.341979 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:32:56.343062 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:32:56.343062 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:32:56.343062 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:32:56.343062 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:32:56.343062 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:32:56.343062 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:32:56.343062 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:32:56.343062 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:32:56.343062 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:32:56.343062 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:32:56.343062 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:32:56.343062 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:32:56.343062 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 593/1195] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 17 23:32:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:32:56.347979 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0fcffb-53a9-4ad6-a8e0-554267679394 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:56.380927 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:56.443449 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0fcffb-53a9-4ad6-a8e0-554267679394 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:56.477046 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6019a6e8-9a9e-4695-8077-648bdaf23172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:56.477817 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019a6e8-9a9e-4695-8077-648bdaf23172 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:56.477936 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019a6e8-9a9e-4695-8077-648bdaf23172 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:56.478190 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019a6e8-9a9e-4695-8077-648bdaf23172 None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:56.478854 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6019a6e8-9a9e-4695-8077-648bdaf23172 None admin] RBAC: Adding query filter params (name=admin) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:32:56.479164 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6019a6e8-9a9e-4695-8077-648bdaf23172 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:56.480582 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6019a6e8-9a9e-4695-8077-648bdaf23172 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:56.488517 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 603/1196] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:32:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:56.494955 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd46f19b-83f0-472a-8cd5-42133ef6bc95 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:56.590019 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd46f19b-83f0-472a-8cd5-42133ef6bc95 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:56.622697 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739d7a75-f68e-4dea-ac6d-8b95979bf1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:56.623294 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d7a75-f68e-4dea-ac6d-8b95979bf1fc None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:56.623567 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d7a75-f68e-4dea-ac6d-8b95979bf1fc None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:56.623834 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d7a75-f68e-4dea-ac6d-8b95979bf1fc None admin] PATH_INFO: `/v3/users/e88ca1ee86e24644b491d1eb34855b77` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:56.624944 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739d7a75-f68e-4dea-ac6d-8b95979bf1fc None admin] RBAC: Authorizing `identity:get_user(user_id=e88ca1ee86e24644b491d1eb34855b77)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:56.626364 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739d7a75-f68e-4dea-ac6d-8b95979bf1fc None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:56.627887 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 594/1197] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Aug 17 23:32:56 2022] GET /identity/v3/users/e88ca1ee86e24644b491d1eb34855b77 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:56.633551 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6019a6e8-9a9e-4695-8077-648bdaf23172 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:56.726201 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:56.730173 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6019a6e8-9a9e-4695-8077-648bdaf23172 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:56.763117 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee150eb6-b9f3-4be5-b20d-64cb9a6a6bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:56.763712 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee150eb6-b9f3-4be5-b20d-64cb9a6a6bbd None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:56.764019 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee150eb6-b9f3-4be5-b20d-64cb9a6a6bbd None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:56.764294 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee150eb6-b9f3-4be5-b20d-64cb9a6a6bbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:56.828570 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee150eb6-b9f3-4be5-b20d-64cb9a6a6bbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:56.830007 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee150eb6-b9f3-4be5-b20d-64cb9a6a6bbd None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:56.921193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:56.925799 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 604/1198] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:32:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 23:32:56.931582 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739d7a75-f68e-4dea-ac6d-8b95979bf1fc None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:56.973557 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:32:57.028649 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739d7a75-f68e-4dea-ac6d-8b95979bf1fc None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:57.060827 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4f4c62-5bc3-490b-8de7-4b80a8cac822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:57.061439 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f4c62-5bc3-490b-8de7-4b80a8cac822 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:57.061719 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f4c62-5bc3-490b-8de7-4b80a8cac822 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:57.061993 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f4c62-5bc3-490b-8de7-4b80a8cac822 None admin] PATH_INFO: `/v3/projects/eb1683a1594744bbb71ad7cb7f3561c2` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:57.063098 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4f4c62-5bc3-490b-8de7-4b80a8cac822 None admin] RBAC: Authorizing `identity:get_project(project_id=eb1683a1594744bbb71ad7cb7f3561c2)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:57.064556 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4f4c62-5bc3-490b-8de7-4b80a8cac822 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:57.066186 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 595/1199] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Aug 17 23:32:56 2022] GET /identity/v3/projects/eb1683a1594744bbb71ad7cb7f3561c2 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:32:57.072069 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee150eb6-b9f3-4be5-b20d-64cb9a6a6bbd None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:32:57.167423 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee150eb6-b9f3-4be5-b20d-64cb9a6a6bbd None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:32:57.199874 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33befee-0841-4ac5-a99d-ab187d7b3130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:32:57.200598 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33befee-0841-4ac5-a99d-ab187d7b3130 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:32:57.200885 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33befee-0841-4ac5-a99d-ab187d7b3130 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:32:57.201160 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33befee-0841-4ac5-a99d-ab187d7b3130 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:32:57.201875 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33befee-0841-4ac5-a99d-ab187d7b3130 None admin] RBAC: Adding query filter params (role.id=33bd60d3b0e94dcbbfe97e87f91a8d78, scope.project.id=eb1683a1594744bbb71ad7cb7f3561c2, user.id=e88ca1ee86e24644b491d1eb34855b77) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:32:57.202189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33befee-0841-4ac5-a99d-ab187d7b3130 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:32:57.203609 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33befee-0841-4ac5-a99d-ab187d7b3130 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:32:57.209186 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 605/1200] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:32:57 2022] GET /identity/v3/role_assignments?user.id=e88ca1ee86e24644b491d1eb34855b77&scope.project.id=eb1683a1594744bbb71ad7cb7f3561c2&role.id=33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:33:07.056771 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=45 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:33:07.206638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=69 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:33:51.818922 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86330127-e8be-4eaf-a98a-585ab1a31edc None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:33:51.819536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86330127-e8be-4eaf-a98a-585ab1a31edc None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:33:51.819536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86330127-e8be-4eaf-a98a-585ab1a31edc None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:33:51.821050 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 596/1201] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:33:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:33:51.826394 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0f47b-b57d-47ab-8de0-152fe23e5961 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:33:51.826875 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0f47b-b57d-47ab-8de0-152fe23e5961 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:33:51.826964 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0f47b-b57d-47ab-8de0-152fe23e5961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:33:51.855376 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-cbb0f47b-b57d-47ab-8de0-152fe23e5961 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:33:51.856646 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-cbb0f47b-b57d-47ab-8de0-152fe23e5961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:33:51.952176 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 606/1202] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:33:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:33:52.241329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86330127-e8be-4eaf-a98a-585ab1a31edc None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:33:52.377218 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86330127-e8be-4eaf-a98a-585ab1a31edc None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:33:52.427541 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2f8740-d972-4cbb-ab24-75f1bb6fd621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:33:52.428288 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f8740-d972-4cbb-ab24-75f1bb6fd621 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:33:52.428644 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f8740-d972-4cbb-ab24-75f1bb6fd621 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:33:52.428969 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f8740-d972-4cbb-ab24-75f1bb6fd621 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:33:52.446421 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2f8740-d972-4cbb-ab24-75f1bb6fd621 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:33:52.448062 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2f8740-d972-4cbb-ab24-75f1bb6fd621 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:33:52.471041 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-5e2f8740-d972-4cbb-ab24-75f1bb6fd621 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:33:52.471041 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:33:52.471041 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:33:52.471041 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:33:52.471041 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:33:52.471041 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:33:52.471041 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:33:52.471041 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:33:52.471041 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:33:52.471041 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:33:52.471041 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:33:52.471041 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:33:52.471041 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:33:52.471041 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:33:52.471041 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:33:52.471041 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:33:52.472224 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:33:52.472224 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:33:52.472224 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:33:52.472224 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:33:52.472224 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:33:52.472224 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:33:52.472224 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:33:52.472224 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:33:52.472224 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:33:52.472224 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:33:52.472224 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:33:52.472224 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:33:52.472224 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:33:52.472224 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:33:52.472224 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:33:52.472224 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:33:52.472224 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:33:52.473245 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:33:52.473245 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:33:52.473245 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:33:52.480473 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 597/1203] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:33:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:33:52.480656 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb0f47b-b57d-47ab-8de0-152fe23e5961 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:33:52.607213 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb0f47b-b57d-47ab-8de0-152fe23e5961 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:33:52.643796 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:33:52.654186 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb0177b-1099-4057-aacb-37ea200f8ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:33:52.654802 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0177b-1099-4057-aacb-37ea200f8ceb None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:33:52.655078 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0177b-1099-4057-aacb-37ea200f8ceb None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:33:52.655348 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0177b-1099-4057-aacb-37ea200f8ceb None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:33:52.656072 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb0177b-1099-4057-aacb-37ea200f8ceb None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:33:52.656513 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb0177b-1099-4057-aacb-37ea200f8ceb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:33:52.657712 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb0177b-1099-4057-aacb-37ea200f8ceb None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:33:52.673263 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 607/1204] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:33:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:33:52.682581 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2f8740-d972-4cbb-ab24-75f1bb6fd621 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:33:52.749064 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:33:52.815327 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2f8740-d972-4cbb-ab24-75f1bb6fd621 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:33:52.858436 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c54d958-1b54-478d-a200-754bd5125b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:33:52.859063 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54d958-1b54-478d-a200-754bd5125b55 None admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:33:52.859288 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54d958-1b54-478d-a200-754bd5125b55 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:33:52.859501 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54d958-1b54-478d-a200-754bd5125b55 None admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:33:52.860177 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c54d958-1b54-478d-a200-754bd5125b55 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:33:52.861647 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c54d958-1b54-478d-a200-754bd5125b55 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:33:52.896188 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 598/1205] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Aug 17 23:33:52 2022] POST /identity/v3/users => generated 251 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:33:55.156076 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa06bd0-63b2-4c00-9e33-c26dfb957477 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:33:55.156639 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa06bd0-63b2-4c00-9e33-c26dfb957477 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:33:55.156639 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa06bd0-63b2-4c00-9e33-c26dfb957477 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:33:55.158125 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 608/1206] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:33:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:33:55.164529 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf56e2-8659-4abf-b94a-6b145d9c36ff None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:33:55.165064 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf56e2-8659-4abf-b94a-6b145d9c36ff None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:33:55.165283 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf56e2-8659-4abf-b94a-6b145d9c36ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:33:55.185479 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-e8cf56e2-8659-4abf-b94a-6b145d9c36ff None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:33:55.186826 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-e8cf56e2-8659-4abf-b94a-6b145d9c36ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:33:55.268434 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 599/1207] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:33:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:33:55.577167 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa06bd0-63b2-4c00-9e33-c26dfb957477 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:33:55.718169 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa06bd0-63b2-4c00-9e33-c26dfb957477 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:33:55.763374 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e7c4a0-86ba-48e1-aba5-4afe23a87cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:33:55.764102 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7c4a0-86ba-48e1-aba5-4afe23a87cfc None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:33:55.764392 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7c4a0-86ba-48e1-aba5-4afe23a87cfc None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:33:55.764708 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7c4a0-86ba-48e1-aba5-4afe23a87cfc None admin] PATH_INFO: `/v3/roles/service` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:33:55.776614 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e7c4a0-86ba-48e1-aba5-4afe23a87cfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:33:55.782086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-f0e7c4a0-86ba-48e1-aba5-4afe23a87cfc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:33:55.782086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:33:55.782086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:33:55.782086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:33:55.782086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:33:55.782086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:33:55.782086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:33:55.782086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:33:55.782086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:33:55.782086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:33:55.782086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:33:55.782086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:33:55.782086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:33:55.782086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:33:55.782086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:33:55.782086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:33:55.785031 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:33:55.785031 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:33:55.785031 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:33:55.785031 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:33:55.785031 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:33:55.785031 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:33:55.785031 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:33:55.785031 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:33:55.785031 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:33:55.785031 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:33:55.785031 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:33:55.785031 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:33:55.785031 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:33:55.785031 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:33:55.785031 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:33:55.785031 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:33:55.786484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:33:55.786484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:33:55.786484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:33:55.786484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:33:55.786484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:33:55.786484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:33:55.786484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:33:55.786484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:33:55.786484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:33:55.786484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:33:55.786484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:33:55.786484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:33:55.786484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 609/1208] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 17 23:33:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:33:55.789797 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf56e2-8659-4abf-b94a-6b145d9c36ff None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:33:55.837492 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:33:55.912616 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf56e2-8659-4abf-b94a-6b145d9c36ff None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:33:55.952902 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8a07e-2615-4012-9b63-d62ead7d514c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:33:55.953607 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8a07e-2615-4012-9b63-d62ead7d514c None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:33:55.953878 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8a07e-2615-4012-9b63-d62ead7d514c None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:33:55.954153 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8a07e-2615-4012-9b63-d62ead7d514c None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:33:55.954880 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d8a07e-2615-4012-9b63-d62ead7d514c None admin] RBAC: Adding query filter params (name=service) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:33:55.955188 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d8a07e-2615-4012-9b63-d62ead7d514c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:33:55.956785 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d8a07e-2615-4012-9b63-d62ead7d514c None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:33:55.966551 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 600/1209] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 17 23:33:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:33:55.974609 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e7c4a0-86ba-48e1-aba5-4afe23a87cfc None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:33:56.032134 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:33:56.097615 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e7c4a0-86ba-48e1-aba5-4afe23a87cfc None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:33:56.138626 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e3526-9ecf-4d99-a705-c15dc6835afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:33:56.139434 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e3526-9ecf-4d99-a705-c15dc6835afc None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:33:56.139807 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e3526-9ecf-4d99-a705-c15dc6835afc None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:33:56.140211 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e3526-9ecf-4d99-a705-c15dc6835afc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:33:56.154280 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e3526-9ecf-4d99-a705-c15dc6835afc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:33:56.156128 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e3526-9ecf-4d99-a705-c15dc6835afc None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:33:56.172776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-501e3526-9ecf-4d99-a705-c15dc6835afc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:33:56.172776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:33:56.172776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:33:56.172776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:33:56.172776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:33:56.172776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:33:56.172776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:33:56.172776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:33:56.172776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:33:56.172776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:33:56.172776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:33:56.172776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:33:56.172776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:33:56.172776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:33:56.172776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:33:56.172776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:33:56.177212 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:33:56.177212 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:33:56.177212 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:33:56.177212 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:33:56.177212 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:33:56.177212 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:33:56.177212 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:33:56.177212 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:33:56.177212 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:33:56.177212 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:33:56.177212 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:33:56.177212 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:33:56.177212 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:33:56.177212 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:33:56.177212 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:33:56.177212 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:33:56.177212 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:33:56.178404 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:33:56.178404 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:33:56.178404 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:33:56.178404 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 610/1210] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:33:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:33:56.180099 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8a07e-2615-4012-9b63-d62ead7d514c None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:33:56.294475 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:33:56.298049 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8a07e-2615-4012-9b63-d62ead7d514c None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:33:56.337760 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00248bfd-4e52-415c-8c77-b5765466a570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:33:56.338596 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00248bfd-4e52-415c-8c77-b5765466a570 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:33:56.338886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00248bfd-4e52-415c-8c77-b5765466a570 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:33:56.339182 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00248bfd-4e52-415c-8c77-b5765466a570 None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:33:56.339993 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00248bfd-4e52-415c-8c77-b5765466a570 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:33:56.340319 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00248bfd-4e52-415c-8c77-b5765466a570 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:33:56.341991 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00248bfd-4e52-415c-8c77-b5765466a570 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:33:56.360056 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 601/1211] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:33:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:33:56.365649 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e3526-9ecf-4d99-a705-c15dc6835afc None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:33:56.480602 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e3526-9ecf-4d99-a705-c15dc6835afc None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:33:56.482082 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:33:56.519680 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c39ce21-5d0b-4caa-b08a-95f379ecc052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:33:56.520906 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c39ce21-5d0b-4caa-b08a-95f379ecc052 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:33:56.521196 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c39ce21-5d0b-4caa-b08a-95f379ecc052 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:33:56.521481 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c39ce21-5d0b-4caa-b08a-95f379ecc052 None admin] PATH_INFO: `/v3/users/placement` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:33:56.529666 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c39ce21-5d0b-4caa-b08a-95f379ecc052 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:33:56.531128 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c39ce21-5d0b-4caa-b08a-95f379ecc052 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:33:56.542334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-4c39ce21-5d0b-4caa-b08a-95f379ecc052 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 17 23:33:56.542334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:33:56.542334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:33:56.542334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:33:56.542334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:33:56.542334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:33:56.542334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:33:56.542334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:33:56.542334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:33:56.542334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:33:56.542334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:33:56.542334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:33:56.542334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 23:33:56.542334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 23:33:56.542334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 23:33:56.542334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 23:33:56.549934 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:33:56.549934 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:33:56.549934 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 23:33:56.549934 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:33:56.549934 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 23:33:56.549934 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:33:56.549934 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:33:56.549934 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:33:56.549934 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:33:56.549934 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:33:56.549934 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:33:56.549934 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:33:56.549934 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:33:56.549934 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:33:56.549934 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:33:56.549934 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:33:56.549934 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:33:56.549934 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:33:56.551105 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:33:56.551105 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:33:56.551105 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 23:33:56.551105 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 23:33:56.551105 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 23:33:56.551105 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 23:33:56.551105 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 23:33:56.551105 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 23:33:56.551105 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 17 23:33:56.551105 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:33:56.551105 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 611/1212] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:33:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:33:56.551742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00248bfd-4e52-415c-8c77-b5765466a570 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:33:56.665338 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00248bfd-4e52-415c-8c77-b5765466a570 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:33:56.703407 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c120aae3-b6c4-4dd5-a2cc-99e1132bc8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:33:56.704067 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120aae3-b6c4-4dd5-a2cc-99e1132bc8e8 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:33:56.704347 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120aae3-b6c4-4dd5-a2cc-99e1132bc8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:33:56.704617 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120aae3-b6c4-4dd5-a2cc-99e1132bc8e8 None admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:33:56.705356 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c120aae3-b6c4-4dd5-a2cc-99e1132bc8e8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:33:56.705658 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c120aae3-b6c4-4dd5-a2cc-99e1132bc8e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:33:56.707134 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c120aae3-b6c4-4dd5-a2cc-99e1132bc8e8 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:33:56.734848 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 602/1213] 10.222.0.26 () {56 vars in 1143 bytes} [Wed Aug 17 23:33:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:33:56.739550 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c39ce21-5d0b-4caa-b08a-95f379ecc052 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:33:56.855114 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c39ce21-5d0b-4caa-b08a-95f379ecc052 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:33:56.899340 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114905f1-935a-4920-8afe-f3e2f31f0052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:33:56.900153 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114905f1-935a-4920-8afe-f3e2f31f0052 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:33:56.902071 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114905f1-935a-4920-8afe-f3e2f31f0052 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:33:56.902411 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114905f1-935a-4920-8afe-f3e2f31f0052 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:33:56.934113 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:33:56.975135 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114905f1-935a-4920-8afe-f3e2f31f0052 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:33:56.976659 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114905f1-935a-4920-8afe-f3e2f31f0052 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:33:57.083075 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 612/1214] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:33:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 23:33:57.088816 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c120aae3-b6c4-4dd5-a2cc-99e1132bc8e8 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:33:57.112549 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:33:57.199914 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c120aae3-b6c4-4dd5-a2cc-99e1132bc8e8 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:33:57.236519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2cfec-99d9-4a33-bc35-94ecd17986a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:33:57.237151 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2cfec-99d9-4a33-bc35-94ecd17986a8 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:33:57.237423 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2cfec-99d9-4a33-bc35-94ecd17986a8 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:33:57.237692 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2cfec-99d9-4a33-bc35-94ecd17986a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:33:57.248616 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de2cfec-99d9-4a33-bc35-94ecd17986a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:33:57.250090 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de2cfec-99d9-4a33-bc35-94ecd17986a8 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:33:57.264974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-6de2cfec-99d9-4a33-bc35-94ecd17986a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:33:57.264974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:33:57.264974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:33:57.264974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:33:57.264974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:33:57.264974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:33:57.264974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:33:57.264974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:33:57.264974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:33:57.264974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:33:57.264974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:33:57.264974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:33:57.264974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:33:57.264974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:33:57.264974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:33:57.264974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:33:57.267117 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:33:57.267117 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:33:57.267117 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:33:57.267117 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:33:57.267117 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:33:57.267117 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:33:57.267117 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:33:57.267117 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:33:57.267117 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:33:57.267117 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:33:57.267117 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:33:57.267117 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:33:57.267117 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:33:57.267117 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:33:57.267117 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:33:57.267117 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:33:57.267117 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:33:57.268186 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:33:57.268186 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:33:57.268186 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:33:57.268186 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 603/1215] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:33:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:33:57.271990 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114905f1-935a-4920-8afe-f3e2f31f0052 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:33:57.334831 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:33:57.380671 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114905f1-935a-4920-8afe-f3e2f31f0052 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:33:57.421663 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7396f-539a-42b2-bd64-27511ce51e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:33:57.422303 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7396f-539a-42b2-bd64-27511ce51e7d None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:33:57.422601 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7396f-539a-42b2-bd64-27511ce51e7d None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:33:57.422870 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7396f-539a-42b2-bd64-27511ce51e7d None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:33:57.423578 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c7396f-539a-42b2-bd64-27511ce51e7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:33:57.423896 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c7396f-539a-42b2-bd64-27511ce51e7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:33:57.425509 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c7396f-539a-42b2-bd64-27511ce51e7d None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:33:57.439565 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 613/1216] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:33:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:33:57.445879 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2cfec-99d9-4a33-bc35-94ecd17986a8 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:33:57.520558 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:33:57.554568 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2cfec-99d9-4a33-bc35-94ecd17986a8 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:33:57.592867 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b06369-bc3b-4615-811c-c56d5290572d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:33:57.592867 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b06369-bc3b-4615-811c-c56d5290572d None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:33:57.592867 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b06369-bc3b-4615-811c-c56d5290572d None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:33:57.592867 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b06369-bc3b-4615-811c-c56d5290572d None admin] PATH_INFO: `/v3/projects/service` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:33:57.600867 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b06369-bc3b-4615-811c-c56d5290572d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:33:57.602290 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b06369-bc3b-4615-811c-c56d5290572d None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:33:57.615618 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-25b06369-bc3b-4615-811c-c56d5290572d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:33:57.615618 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:33:57.615618 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:33:57.615618 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:33:57.615618 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:33:57.615618 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:33:57.615618 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:33:57.615618 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:33:57.615618 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:33:57.615618 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:33:57.615618 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:33:57.615618 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:33:57.615618 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 23:33:57.615618 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 23:33:57.615618 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 23:33:57.616791 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 23:33:57.616791 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:33:57.616791 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:33:57.616791 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:33:57.616791 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:33:57.616791 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:33:57.616791 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:33:57.616791 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:33:57.616791 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:33:57.616791 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:33:57.616791 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:33:57.616791 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:33:57.616791 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:33:57.616791 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:33:57.616791 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:33:57.616791 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:33:57.616791 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:33:57.617793 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 23:33:57.617793 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 23:33:57.617793 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 23:33:57.617793 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 23:33:57.617793 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 23:33:57.617793 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 23:33:57.617793 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:33:57.617793 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:33:57.617793 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 604/1217] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:33:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:33:57.623098 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7396f-539a-42b2-bd64-27511ce51e7d None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:33:57.739530 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7396f-539a-42b2-bd64-27511ce51e7d None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:33:57.765814 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:33:57.780620 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e8dcb4-4c76-4318-b30f-af7572f9f802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:33:57.780620 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8dcb4-4c76-4318-b30f-af7572f9f802 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:33:57.780901 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8dcb4-4c76-4318-b30f-af7572f9f802 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:33:57.781142 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8dcb4-4c76-4318-b30f-af7572f9f802 None admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:33:57.781829 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e8dcb4-4c76-4318-b30f-af7572f9f802 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:33:57.782132 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e8dcb4-4c76-4318-b30f-af7572f9f802 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:33:57.783541 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e8dcb4-4c76-4318-b30f-af7572f9f802 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:33:57.796794 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 614/1218] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 17 23:33:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:33:57.802931 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b06369-bc3b-4615-811c-c56d5290572d None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:33:57.917016 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b06369-bc3b-4615-811c-c56d5290572d None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:33:57.948202 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:33:57.955301 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98a9862-5390-4e04-a0e9-a2535e61b46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:33:57.956069 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a9862-5390-4e04-a0e9-a2535e61b46d None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:33:57.956219 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a9862-5390-4e04-a0e9-a2535e61b46d None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:33:57.956497 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a9862-5390-4e04-a0e9-a2535e61b46d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:33:57.957245 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98a9862-5390-4e04-a0e9-a2535e61b46d None admin] RBAC: Adding query filter params (role.id=b64f80e6f3ea4ad49b94acdde42ccd87, scope.project.id=4de86915ef4340248ba8a8f51d56594e, user.id=1fc57d911a56423e96c736bc96270ba8) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:33:57.958080 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98a9862-5390-4e04-a0e9-a2535e61b46d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:33:57.959525 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98a9862-5390-4e04-a0e9-a2535e61b46d None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:33:57.965482 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 605/1219] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:33:57 2022] GET /identity/v3/role_assignments?user.id=1fc57d911a56423e96c736bc96270ba8&scope.project.id=4de86915ef4340248ba8a8f51d56594e&role.id=b64f80e6f3ea4ad49b94acdde42ccd87 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:33:59.957895 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a17f37-f41f-44ef-abc1-bf88b16f879b None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:33:59.958424 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a17f37-f41f-44ef-abc1-bf88b16f879b None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:33:59.958424 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a17f37-f41f-44ef-abc1-bf88b16f879b None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:33:59.959814 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 615/1220] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:33:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 23:33:59.965675 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24281d0-f5fa-41b9-a220-bf1e47baf772 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:33:59.966103 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24281d0-f5fa-41b9-a220-bf1e47baf772 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:33:59.966215 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24281d0-f5fa-41b9-a220-bf1e47baf772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:33:59.984026 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-d24281d0-f5fa-41b9-a220-bf1e47baf772 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:33:59.985193 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-d24281d0-f5fa-41b9-a220-bf1e47baf772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:34:00.055858 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 606/1221] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:33:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:34:00.338382 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a17f37-f41f-44ef-abc1-bf88b16f879b None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:00.452239 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a17f37-f41f-44ef-abc1-bf88b16f879b None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:00.491102 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea7b055-eb8a-4490-ade2-e2835d91a89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:00.491729 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7b055-eb8a-4490-ade2-e2835d91a89d None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:00.492038 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7b055-eb8a-4490-ade2-e2835d91a89d None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:00.492312 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7b055-eb8a-4490-ade2-e2835d91a89d None admin] PATH_INFO: `/v3/roles/service` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:00.504132 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea7b055-eb8a-4490-ade2-e2835d91a89d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:34:00.509848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-4ea7b055-eb8a-4490-ade2-e2835d91a89d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:34:00.509848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:34:00.509848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:34:00.509848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:34:00.509848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:34:00.509848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:34:00.509848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:34:00.509848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:34:00.509848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:34:00.509848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:34:00.509848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:34:00.509848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:34:00.509848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:34:00.509848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:34:00.509848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:34:00.509848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:34:00.511012 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:34:00.511012 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:34:00.511012 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:34:00.511012 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:34:00.511012 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:34:00.511012 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:34:00.511012 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:34:00.511012 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:34:00.511012 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:34:00.511012 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:34:00.511012 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:34:00.511012 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:34:00.511012 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:34:00.511012 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:34:00.511012 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:34:00.511012 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:34:00.512218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:34:00.512218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:34:00.512218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:34:00.512218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:34:00.512218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:34:00.512218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:34:00.512218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:34:00.512218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:34:00.512218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:34:00.512218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:34:00.512218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:34:00.512218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:34:00.512218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 616/1222] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 17 23:34:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:34:00.516973 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24281d0-f5fa-41b9-a220-bf1e47baf772 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:00.624480 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24281d0-f5fa-41b9-a220-bf1e47baf772 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:00.650605 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:00.665476 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d607685-2ddd-4871-9f29-f3b4c62c0cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:00.666089 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d607685-2ddd-4871-9f29-f3b4c62c0cfa None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:00.666383 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d607685-2ddd-4871-9f29-f3b4c62c0cfa None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:00.666651 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d607685-2ddd-4871-9f29-f3b4c62c0cfa None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:00.667314 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d607685-2ddd-4871-9f29-f3b4c62c0cfa None admin] RBAC: Adding query filter params (name=service) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:34:00.667623 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d607685-2ddd-4871-9f29-f3b4c62c0cfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:00.669031 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d607685-2ddd-4871-9f29-f3b4c62c0cfa None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:00.677936 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 607/1223] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 17 23:34:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:34:00.684188 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea7b055-eb8a-4490-ade2-e2835d91a89d None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:00.737235 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:00.802071 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea7b055-eb8a-4490-ade2-e2835d91a89d None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:00.837544 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af058a-e012-4a24-985a-cfeede099120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:00.838175 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af058a-e012-4a24-985a-cfeede099120 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:00.838456 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af058a-e012-4a24-985a-cfeede099120 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:00.838725 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af058a-e012-4a24-985a-cfeede099120 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:00.850006 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7af058a-e012-4a24-985a-cfeede099120 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:00.851623 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7af058a-e012-4a24-985a-cfeede099120 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:00.866621 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-a7af058a-e012-4a24-985a-cfeede099120 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:34:00.866621 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:34:00.866621 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:34:00.866621 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:34:00.866621 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:34:00.866621 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:34:00.866621 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:34:00.866621 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:34:00.866621 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:34:00.866621 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:34:00.866621 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:34:00.866621 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:34:00.866621 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:34:00.866621 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:34:00.866621 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:34:00.866621 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:34:00.867682 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:34:00.867682 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:34:00.867682 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:34:00.867682 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:34:00.867682 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:34:00.867682 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:34:00.867682 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:34:00.867682 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:34:00.867682 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:34:00.867682 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:34:00.867682 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:34:00.867682 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:34:00.867682 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:34:00.867682 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:34:00.867682 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:34:00.867682 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:34:00.867682 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:34:00.873344 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:34:00.873344 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:34:00.873344 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:34:00.873344 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 617/1224] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:34:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:34:00.873705 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d607685-2ddd-4871-9f29-f3b4c62c0cfa None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:00.984156 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d607685-2ddd-4871-9f29-f3b4c62c0cfa None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:01.025100 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75402a51-608f-4957-a820-f7bcc07c6dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:01.025872 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75402a51-608f-4957-a820-f7bcc07c6dfc None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:01.026158 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75402a51-608f-4957-a820-f7bcc07c6dfc None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:01.026432 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75402a51-608f-4957-a820-f7bcc07c6dfc None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:01.027087 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75402a51-608f-4957-a820-f7bcc07c6dfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:34:01.027383 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75402a51-608f-4957-a820-f7bcc07c6dfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:01.029059 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75402a51-608f-4957-a820-f7bcc07c6dfc None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:01.041292 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 608/1225] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:34:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:34:01.047038 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af058a-e012-4a24-985a-cfeede099120 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:01.143741 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:01.154133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af058a-e012-4a24-985a-cfeede099120 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:01.190641 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588de146-785b-4069-8a77-e4b301e52bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:01.191267 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588de146-785b-4069-8a77-e4b301e52bbf None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:01.191541 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588de146-785b-4069-8a77-e4b301e52bbf None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:01.191853 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588de146-785b-4069-8a77-e4b301e52bbf None admin] PATH_INFO: `/v3/users/placement` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:01.199341 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588de146-785b-4069-8a77-e4b301e52bbf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:01.200832 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588de146-785b-4069-8a77-e4b301e52bbf None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:01.211768 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-588de146-785b-4069-8a77-e4b301e52bbf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 17 23:34:01.211768 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:34:01.211768 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:34:01.211768 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:34:01.211768 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:34:01.211768 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:34:01.211768 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:34:01.211768 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:34:01.211768 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:34:01.211768 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:34:01.211768 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:34:01.211768 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:34:01.211768 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 23:34:01.211768 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 23:34:01.211768 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 23:34:01.211768 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 23:34:01.212837 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:34:01.212837 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:34:01.212837 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 23:34:01.212837 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:34:01.212837 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 23:34:01.212837 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:34:01.212837 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:34:01.212837 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:34:01.212837 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:34:01.212837 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:34:01.212837 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:34:01.212837 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:34:01.212837 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:34:01.212837 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:34:01.212837 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:34:01.212837 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:34:01.212837 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:34:01.212837 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:34:01.213796 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:34:01.213796 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:34:01.213796 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 23:34:01.213796 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 23:34:01.213796 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 23:34:01.213796 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 23:34:01.213796 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 23:34:01.213796 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 23:34:01.213796 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 17 23:34:01.213796 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:34:01.213796 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 618/1226] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:34:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:34:01.219069 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75402a51-608f-4957-a820-f7bcc07c6dfc None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:01.269280 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:01.325293 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75402a51-608f-4957-a820-f7bcc07c6dfc None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:01.360452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481022c3-c0b0-49d9-8c6e-fce1061978ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:01.361336 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481022c3-c0b0-49d9-8c6e-fce1061978ff None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:01.361617 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481022c3-c0b0-49d9-8c6e-fce1061978ff None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:01.361895 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481022c3-c0b0-49d9-8c6e-fce1061978ff None admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:01.362619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481022c3-c0b0-49d9-8c6e-fce1061978ff None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:34:01.362968 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481022c3-c0b0-49d9-8c6e-fce1061978ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:01.365063 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481022c3-c0b0-49d9-8c6e-fce1061978ff None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:01.387073 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 609/1227] 10.222.0.26 () {56 vars in 1143 bytes} [Wed Aug 17 23:34:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:34:01.388300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588de146-785b-4069-8a77-e4b301e52bbf None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:01.506153 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588de146-785b-4069-8a77-e4b301e52bbf None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:01.542017 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a687c-ca12-4f82-a01d-c8ef3c033d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:01.542749 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a687c-ca12-4f82-a01d-c8ef3c033d16 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:01.546557 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a687c-ca12-4f82-a01d-c8ef3c033d16 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:01.546557 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a687c-ca12-4f82-a01d-c8ef3c033d16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:01.554476 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726a687c-ca12-4f82-a01d-c8ef3c033d16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:01.555955 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726a687c-ca12-4f82-a01d-c8ef3c033d16 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:01.570057 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-726a687c-ca12-4f82-a01d-c8ef3c033d16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:34:01.570057 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:34:01.570057 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:34:01.570057 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:34:01.570057 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:34:01.570057 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:34:01.570057 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:34:01.570057 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:34:01.570057 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:34:01.570057 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:34:01.570057 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:34:01.570057 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:34:01.570057 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:34:01.570057 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:34:01.570057 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:34:01.570057 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:34:01.571043 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:34:01.571043 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:34:01.571043 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:34:01.571043 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:34:01.571043 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:34:01.571043 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:34:01.571043 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:34:01.571043 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:34:01.571043 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:34:01.571043 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:34:01.571043 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:34:01.571043 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:34:01.571043 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:34:01.571043 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:34:01.571043 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:34:01.571043 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:34:01.571043 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:34:01.572168 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:34:01.572168 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:34:01.572168 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:34:01.572168 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 619/1228] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:34:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:34:01.577173 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481022c3-c0b0-49d9-8c6e-fce1061978ff None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:01.684959 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481022c3-c0b0-49d9-8c6e-fce1061978ff None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:01.687507 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:01.724062 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7933e1-ce65-4e74-bfbe-147d35dc619d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:01.724687 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7933e1-ce65-4e74-bfbe-147d35dc619d None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:01.724969 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7933e1-ce65-4e74-bfbe-147d35dc619d None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:01.725245 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7933e1-ce65-4e74-bfbe-147d35dc619d None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:01.725890 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7933e1-ce65-4e74-bfbe-147d35dc619d None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:34:01.726194 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7933e1-ce65-4e74-bfbe-147d35dc619d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:01.727586 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7933e1-ce65-4e74-bfbe-147d35dc619d None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:01.740764 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 610/1229] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:34:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:34:01.746773 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a687c-ca12-4f82-a01d-c8ef3c033d16 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:01.758470 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:01.858720 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a687c-ca12-4f82-a01d-c8ef3c033d16 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:01.895978 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2a5cbc-d09f-40bc-b66d-e41f50ce68b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:01.896766 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a5cbc-d09f-40bc-b66d-e41f50ce68b6 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:01.897116 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a5cbc-d09f-40bc-b66d-e41f50ce68b6 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:01.897343 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a5cbc-d09f-40bc-b66d-e41f50ce68b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:01.905598 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2a5cbc-d09f-40bc-b66d-e41f50ce68b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:01.907118 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2a5cbc-d09f-40bc-b66d-e41f50ce68b6 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:01.918998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-6f2a5cbc-d09f-40bc-b66d-e41f50ce68b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:34:01.918998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:34:01.918998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:34:01.918998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:34:01.918998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:34:01.918998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:34:01.918998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:34:01.918998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:34:01.918998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:34:01.918998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:34:01.918998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:34:01.918998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:34:01.918998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 23:34:01.918998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 23:34:01.918998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 23:34:01.920133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 23:34:01.920133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:34:01.920133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:34:01.920133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:34:01.920133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:34:01.920133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:34:01.920133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:34:01.920133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:34:01.920133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:34:01.920133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:34:01.920133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:34:01.920133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:34:01.920133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:34:01.920133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:34:01.920133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:34:01.920133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:34:01.920133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:34:01.921083 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 23:34:01.921083 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 23:34:01.921083 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 23:34:01.921083 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 23:34:01.921083 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 23:34:01.921083 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 23:34:01.921083 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:34:01.921083 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:34:01.921083 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 620/1230] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:34:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:34:01.926094 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7933e1-ce65-4e74-bfbe-147d35dc619d None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:02.032237 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7933e1-ce65-4e74-bfbe-147d35dc619d None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:02.070932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4f674-21c8-4179-a39e-2425d02ee6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:02.071587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4f674-21c8-4179-a39e-2425d02ee6a3 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:02.071888 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4f674-21c8-4179-a39e-2425d02ee6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:02.072366 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4f674-21c8-4179-a39e-2425d02ee6a3 None admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:02.072986 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be4f674-21c8-4179-a39e-2425d02ee6a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:34:02.073297 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be4f674-21c8-4179-a39e-2425d02ee6a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:02.074894 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be4f674-21c8-4179-a39e-2425d02ee6a3 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:02.088297 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 611/1231] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 17 23:34:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:34:02.094210 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2a5cbc-d09f-40bc-b66d-e41f50ce68b6 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:02.167844 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:02.208346 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2a5cbc-d09f-40bc-b66d-e41f50ce68b6 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:02.243447 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13108640-a927-419e-9962-d61b4066a201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:02.244167 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13108640-a927-419e-9962-d61b4066a201 None admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:02.244446 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13108640-a927-419e-9962-d61b4066a201 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:02.244737 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13108640-a927-419e-9962-d61b4066a201 None admin] PATH_INFO: `/v3/projects/4de86915ef4340248ba8a8f51d56594e/users/1fc57d911a56423e96c736bc96270ba8/roles/b64f80e6f3ea4ad49b94acdde42ccd87` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:02.260815 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13108640-a927-419e-9962-d61b4066a201 None admin] RBAC: Authorizing `identity:create_grant(project_id=4de86915ef4340248ba8a8f51d56594e, user_id=1fc57d911a56423e96c736bc96270ba8, role_id=b64f80e6f3ea4ad49b94acdde42ccd87)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:02.262320 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13108640-a927-419e-9962-d61b4066a201 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:02.275333 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 621/1232] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 17 23:34:02 2022] PUT /identity/v3/projects/4de86915ef4340248ba8a8f51d56594e/users/1fc57d911a56423e96c736bc96270ba8/roles/b64f80e6f3ea4ad49b94acdde42ccd87 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:34:04.212314 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cac079-3b8b-4c30-bbb0-7f9b76de4491 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:04.212980 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cac079-3b8b-4c30-bbb0-7f9b76de4491 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:04.212980 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cac079-3b8b-4c30-bbb0-7f9b76de4491 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:04.214303 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 612/1233] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:34:04.219861 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32439b5e-1956-4ceb-9711-3400979c1ce7 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:04.220309 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32439b5e-1956-4ceb-9711-3400979c1ce7 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:04.220476 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32439b5e-1956-4ceb-9711-3400979c1ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:04.239777 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-32439b5e-1956-4ceb-9711-3400979c1ce7 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:34:04.241068 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-32439b5e-1956-4ceb-9711-3400979c1ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:34:04.309163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 622/1234] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:34:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:34:04.586965 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cac079-3b8b-4c30-bbb0-7f9b76de4491 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:04.614903 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:04.700217 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cac079-3b8b-4c30-bbb0-7f9b76de4491 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:04.742416 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061801d6-e92d-46de-a5c1-5c65bb81a4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:04.743167 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061801d6-e92d-46de-a5c1-5c65bb81a4a1 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:04.743489 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061801d6-e92d-46de-a5c1-5c65bb81a4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:04.743778 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061801d6-e92d-46de-a5c1-5c65bb81a4a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:04.758101 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061801d6-e92d-46de-a5c1-5c65bb81a4a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:34:04.765233 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-061801d6-e92d-46de-a5c1-5c65bb81a4a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:34:04.765233 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:34:04.765233 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:34:04.765233 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:34:04.765233 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:34:04.765233 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:34:04.765233 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:34:04.765233 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:34:04.765233 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:34:04.765233 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:34:04.765233 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:34:04.765233 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:34:04.765233 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:34:04.765233 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:34:04.765233 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:34:04.765233 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:34:04.766574 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:34:04.766574 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:34:04.766574 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:34:04.766574 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:34:04.766574 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:34:04.766574 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:34:04.766574 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:34:04.766574 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:34:04.766574 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:34:04.766574 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:34:04.766574 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:34:04.766574 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:34:04.766574 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:34:04.766574 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:34:04.766574 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:34:04.766574 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:34:04.767638 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:34:04.767638 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:34:04.767638 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:34:04.767638 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:34:04.767638 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:34:04.767638 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:34:04.767638 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:34:04.767638 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:34:04.767638 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:34:04.767638 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:34:04.767638 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 23:34:04.767638 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:34:04.767638 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 613/1235] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Aug 17 23:34:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:34:04.777438 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32439b5e-1956-4ceb-9711-3400979c1ce7 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:04.830679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:04.890190 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32439b5e-1956-4ceb-9711-3400979c1ce7 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:04.929082 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e5420-bda8-4fb2-b914-776731b1383b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:04.929787 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e5420-bda8-4fb2-b914-776731b1383b None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:04.930177 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e5420-bda8-4fb2-b914-776731b1383b None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:04.930481 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e5420-bda8-4fb2-b914-776731b1383b None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:04.931264 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775e5420-bda8-4fb2-b914-776731b1383b None admin] RBAC: Adding query filter params (name=service) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:34:04.931592 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775e5420-bda8-4fb2-b914-776731b1383b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:04.933099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775e5420-bda8-4fb2-b914-776731b1383b None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:04.942760 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 623/1236] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Aug 17 23:34:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:34:04.948476 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061801d6-e92d-46de-a5c1-5c65bb81a4a1 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:05.045606 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:05.064522 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061801d6-e92d-46de-a5c1-5c65bb81a4a1 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:05.101213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae43e9-1482-4e14-b15b-5939880e93b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:05.101921 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae43e9-1482-4e14-b15b-5939880e93b3 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:05.102227 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae43e9-1482-4e14-b15b-5939880e93b3 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:05.102537 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae43e9-1482-4e14-b15b-5939880e93b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:05.114940 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae43e9-1482-4e14-b15b-5939880e93b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:05.116508 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae43e9-1482-4e14-b15b-5939880e93b3 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:05.131407 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-82ae43e9-1482-4e14-b15b-5939880e93b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:34:05.131407 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:34:05.131407 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:34:05.131407 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:34:05.131407 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:34:05.131407 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:34:05.131407 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:34:05.131407 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:34:05.131407 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:34:05.131407 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:34:05.131407 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:34:05.131407 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:34:05.131407 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:34:05.131407 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:34:05.131407 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:34:05.131407 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:34:05.132615 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:34:05.132615 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:34:05.132615 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:34:05.132615 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:34:05.132615 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:34:05.132615 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:34:05.132615 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:34:05.132615 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:34:05.132615 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:34:05.132615 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:34:05.132615 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:34:05.132615 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:34:05.132615 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:34:05.132615 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:34:05.132615 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:34:05.132615 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:34:05.132615 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:34:05.133744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:34:05.133744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:34:05.133744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:34:05.133744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 614/1237] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:34:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 23:34:05.138524 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e5420-bda8-4fb2-b914-776731b1383b None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:05.246960 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e5420-bda8-4fb2-b914-776731b1383b None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:05.257852 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:05.286310 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b9123-f27c-43cd-9513-c77f5c01d14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:05.287055 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b9123-f27c-43cd-9513-c77f5c01d14f None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:05.287334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b9123-f27c-43cd-9513-c77f5c01d14f None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:05.287639 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b9123-f27c-43cd-9513-c77f5c01d14f None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:05.288457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b9123-f27c-43cd-9513-c77f5c01d14f None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:34:05.288853 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b9123-f27c-43cd-9513-c77f5c01d14f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:05.290363 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b9123-f27c-43cd-9513-c77f5c01d14f None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:05.304266 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 624/1238] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:34:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:34:05.310819 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae43e9-1482-4e14-b15b-5939880e93b3 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:05.420536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae43e9-1482-4e14-b15b-5939880e93b3 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:05.458343 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da853bb3-aca5-4a99-9d16-9e82f97031a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:05.458965 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da853bb3-aca5-4a99-9d16-9e82f97031a2 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:05.459246 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da853bb3-aca5-4a99-9d16-9e82f97031a2 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:05.459561 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da853bb3-aca5-4a99-9d16-9e82f97031a2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:05.462681 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:05.469423 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da853bb3-aca5-4a99-9d16-9e82f97031a2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:05.471039 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da853bb3-aca5-4a99-9d16-9e82f97031a2 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:05.483328 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-da853bb3-aca5-4a99-9d16-9e82f97031a2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 17 23:34:05.483328 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:34:05.483328 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:34:05.483328 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:34:05.483328 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:34:05.483328 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:34:05.483328 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:34:05.483328 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:34:05.483328 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:34:05.483328 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:34:05.483328 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:34:05.483328 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:34:05.483328 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 23:34:05.483328 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 23:34:05.483328 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 23:34:05.483328 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 23:34:05.484482 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:34:05.484482 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:34:05.484482 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 23:34:05.484482 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:34:05.484482 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 23:34:05.484482 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:34:05.484482 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:34:05.484482 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:34:05.484482 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:34:05.484482 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:34:05.484482 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:34:05.484482 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:34:05.484482 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:34:05.484482 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:34:05.484482 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:34:05.484482 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:34:05.484482 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:34:05.484482 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:34:05.485644 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:34:05.485644 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:34:05.485644 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 23:34:05.485644 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 23:34:05.485644 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 23:34:05.485644 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 23:34:05.485644 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 23:34:05.485644 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 23:34:05.485644 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 17 23:34:05.485644 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:34:05.485644 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 615/1239] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:34:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:34:05.490982 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b9123-f27c-43cd-9513-c77f5c01d14f None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:05.600918 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b9123-f27c-43cd-9513-c77f5c01d14f None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:05.638325 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e360f8ef-7975-4293-901a-ab9013e2514c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:05.638942 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360f8ef-7975-4293-901a-ab9013e2514c None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:05.639259 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360f8ef-7975-4293-901a-ab9013e2514c None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:05.639500 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360f8ef-7975-4293-901a-ab9013e2514c None admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:05.640533 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e360f8ef-7975-4293-901a-ab9013e2514c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:34:05.640826 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e360f8ef-7975-4293-901a-ab9013e2514c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:05.642271 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e360f8ef-7975-4293-901a-ab9013e2514c None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:05.662345 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 625/1240] 10.222.0.26 () {56 vars in 1143 bytes} [Wed Aug 17 23:34:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:34:05.668366 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da853bb3-aca5-4a99-9d16-9e82f97031a2 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:05.796035 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da853bb3-aca5-4a99-9d16-9e82f97031a2 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:05.842131 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46dcd26-5a40-42da-8961-eb88a40167ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:05.842766 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46dcd26-5a40-42da-8961-eb88a40167ba None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:05.843056 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46dcd26-5a40-42da-8961-eb88a40167ba None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:05.843443 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46dcd26-5a40-42da-8961-eb88a40167ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:05.913219 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:05.921280 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46dcd26-5a40-42da-8961-eb88a40167ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:05.922864 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46dcd26-5a40-42da-8961-eb88a40167ba None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:06.036692 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 616/1241] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:34:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 23:34:06.042849 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e360f8ef-7975-4293-901a-ab9013e2514c None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:06.073894 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:06.172463 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e360f8ef-7975-4293-901a-ab9013e2514c None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:06.211348 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbad117-36e2-45ca-b6a5-17dddb922964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:06.212015 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbad117-36e2-45ca-b6a5-17dddb922964 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:06.212303 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbad117-36e2-45ca-b6a5-17dddb922964 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:06.212577 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbad117-36e2-45ca-b6a5-17dddb922964 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:06.225595 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbad117-36e2-45ca-b6a5-17dddb922964 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:06.227064 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbad117-36e2-45ca-b6a5-17dddb922964 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:06.244033 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-bfbad117-36e2-45ca-b6a5-17dddb922964 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:34:06.244033 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:34:06.244033 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:34:06.244033 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:34:06.244033 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:34:06.244033 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:34:06.244033 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:34:06.244033 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:34:06.244033 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:34:06.244033 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:34:06.244033 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:34:06.244033 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:34:06.244033 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:34:06.244033 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:34:06.244033 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:34:06.244033 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:34:06.245099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:34:06.245099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:34:06.245099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:34:06.245099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:34:06.245099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:34:06.245099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:34:06.245099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:34:06.245099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:34:06.245099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:34:06.245099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:34:06.245099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:34:06.245099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:34:06.245099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:34:06.245099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:34:06.245099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:34:06.245099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:34:06.245099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:34:06.251050 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:34:06.251050 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:34:06.251050 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:34:06.251050 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 626/1242] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:34:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:34:06.251314 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46dcd26-5a40-42da-8961-eb88a40167ba None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:06.353308 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:06.360609 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46dcd26-5a40-42da-8961-eb88a40167ba None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:06.398078 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e7d748-4df2-4428-ba91-d5b4347cbc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:06.398689 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e7d748-4df2-4428-ba91-d5b4347cbc9e None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:06.398969 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e7d748-4df2-4428-ba91-d5b4347cbc9e None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:06.399236 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e7d748-4df2-4428-ba91-d5b4347cbc9e None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:06.399885 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e7d748-4df2-4428-ba91-d5b4347cbc9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:34:06.400252 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e7d748-4df2-4428-ba91-d5b4347cbc9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:06.401754 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e7d748-4df2-4428-ba91-d5b4347cbc9e None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:06.415871 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 617/1243] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:34:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:34:06.421882 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbad117-36e2-45ca-b6a5-17dddb922964 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:06.530959 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:06.556665 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbad117-36e2-45ca-b6a5-17dddb922964 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:06.594650 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d2c10-93be-412e-9539-9eb6e86ad889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:06.595286 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d2c10-93be-412e-9539-9eb6e86ad889 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:06.595580 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d2c10-93be-412e-9539-9eb6e86ad889 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:06.595843 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d2c10-93be-412e-9539-9eb6e86ad889 None admin] PATH_INFO: `/v3/projects/service` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:06.602809 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1d2c10-93be-412e-9539-9eb6e86ad889 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:06.604318 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1d2c10-93be-412e-9539-9eb6e86ad889 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:06.617551 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-1e1d2c10-93be-412e-9539-9eb6e86ad889 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:34:06.617551 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:34:06.617551 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:34:06.617551 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:34:06.617551 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:34:06.617551 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:34:06.617551 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:34:06.617551 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:34:06.617551 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:34:06.617551 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:34:06.617551 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:34:06.617551 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:34:06.617551 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 23:34:06.617551 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 23:34:06.617551 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 23:34:06.618801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 23:34:06.618801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:34:06.618801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:34:06.618801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:34:06.618801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:34:06.618801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:34:06.618801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:34:06.618801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:34:06.618801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:34:06.618801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:34:06.618801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:34:06.618801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:34:06.618801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:34:06.618801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:34:06.618801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:34:06.618801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:34:06.618801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:34:06.619902 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 23:34:06.619902 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 23:34:06.619902 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 23:34:06.619902 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 23:34:06.619902 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 23:34:06.619902 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 23:34:06.619902 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:34:06.619902 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:34:06.619902 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 627/1244] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:34:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:34:06.628040 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e7d748-4df2-4428-ba91-d5b4347cbc9e None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:06.735683 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e7d748-4df2-4428-ba91-d5b4347cbc9e None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:06.776774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a99c31-3c66-4715-aef7-46537a28aff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:06.777398 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a99c31-3c66-4715-aef7-46537a28aff4 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:06.778053 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a99c31-3c66-4715-aef7-46537a28aff4 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:06.778441 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a99c31-3c66-4715-aef7-46537a28aff4 None admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:06.779336 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a99c31-3c66-4715-aef7-46537a28aff4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:34:06.779726 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a99c31-3c66-4715-aef7-46537a28aff4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:06.781353 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a99c31-3c66-4715-aef7-46537a28aff4 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:06.796041 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 618/1245] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 17 23:34:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:34:06.802295 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d2c10-93be-412e-9539-9eb6e86ad889 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:06.912115 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d2c10-93be-412e-9539-9eb6e86ad889 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:06.946762 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519820f9-9a53-42f6-b80f-ae834b457af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:06.947372 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519820f9-9a53-42f6-b80f-ae834b457af6 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:06.947653 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519820f9-9a53-42f6-b80f-ae834b457af6 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:06.947926 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519820f9-9a53-42f6-b80f-ae834b457af6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:06.948678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519820f9-9a53-42f6-b80f-ae834b457af6 None admin] RBAC: Adding query filter params (role.id=b64f80e6f3ea4ad49b94acdde42ccd87, scope.project.id=4de86915ef4340248ba8a8f51d56594e, user.id=1fc57d911a56423e96c736bc96270ba8) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:34:06.948995 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519820f9-9a53-42f6-b80f-ae834b457af6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:06.950491 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519820f9-9a53-42f6-b80f-ae834b457af6 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:06.956332 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 628/1246] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:34:06 2022] GET /identity/v3/role_assignments?user.id=1fc57d911a56423e96c736bc96270ba8&scope.project.id=4de86915ef4340248ba8a8f51d56594e&role.id=b64f80e6f3ea4ad49b94acdde42ccd87 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:34:08.980952 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fcb847-374c-4193-8be1-ffd8fc41eb1d None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:08.981495 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fcb847-374c-4193-8be1-ffd8fc41eb1d None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:08.981495 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fcb847-374c-4193-8be1-ffd8fc41eb1d None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:08.982823 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 619/1247] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:34:08.988469 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12bd0ce-5aea-41ef-8c1f-0897aa6446b4 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:08.988913 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12bd0ce-5aea-41ef-8c1f-0897aa6446b4 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:08.989115 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12bd0ce-5aea-41ef-8c1f-0897aa6446b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:08.995766 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:09.009186 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-a12bd0ce-5aea-41ef-8c1f-0897aa6446b4 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:34:09.010578 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-a12bd0ce-5aea-41ef-8c1f-0897aa6446b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:34:09.085215 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 629/1248] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:34:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 23:34:09.370622 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fcb847-374c-4193-8be1-ffd8fc41eb1d None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:09.389528 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:09.483912 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fcb847-374c-4193-8be1-ffd8fc41eb1d None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:09.525840 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66569ede-5da7-4388-b71b-b74571989c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:09.526669 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66569ede-5da7-4388-b71b-b74571989c65 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:09.527139 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66569ede-5da7-4388-b71b-b74571989c65 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:09.527502 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66569ede-5da7-4388-b71b-b74571989c65 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:09.539507 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66569ede-5da7-4388-b71b-b74571989c65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:34:09.544301 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-66569ede-5da7-4388-b71b-b74571989c65 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:34:09.544301 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:34:09.544301 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:34:09.544301 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:34:09.544301 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:34:09.544301 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:34:09.544301 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:34:09.544301 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:34:09.544301 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:34:09.544301 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:34:09.544301 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:34:09.544301 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:34:09.544301 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:34:09.544301 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:34:09.544301 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:34:09.544301 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:34:09.545520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:34:09.545520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:34:09.545520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:34:09.545520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:34:09.545520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:34:09.545520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:34:09.545520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:34:09.545520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:34:09.545520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:34:09.545520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:34:09.545520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:34:09.545520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:34:09.545520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:34:09.545520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:34:09.545520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:34:09.545520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:34:09.546567 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:34:09.546567 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:34:09.546567 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:34:09.546567 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:34:09.546567 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:34:09.546567 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:34:09.546567 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:34:09.546567 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:34:09.546567 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:34:09.546567 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:34:09.546567 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:34:09.546567 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:34:09.546567 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 620/1249] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 17 23:34:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:34:09.551401 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12bd0ce-5aea-41ef-8c1f-0897aa6446b4 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:09.655985 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12bd0ce-5aea-41ef-8c1f-0897aa6446b4 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:09.700457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:09.708010 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5293ea88-5e50-4e99-b5b2-85bc9243c291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:09.708010 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293ea88-5e50-4e99-b5b2-85bc9243c291 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:09.708010 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293ea88-5e50-4e99-b5b2-85bc9243c291 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:09.708482 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293ea88-5e50-4e99-b5b2-85bc9243c291 None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:09.709191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5293ea88-5e50-4e99-b5b2-85bc9243c291 None admin] RBAC: Adding query filter params (name=admin) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:34:09.709899 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5293ea88-5e50-4e99-b5b2-85bc9243c291 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:09.711276 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5293ea88-5e50-4e99-b5b2-85bc9243c291 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:09.720220 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 630/1250] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:34:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:34:09.727015 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66569ede-5da7-4388-b71b-b74571989c65 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:09.829997 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:09.859653 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66569ede-5da7-4388-b71b-b74571989c65 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:09.901461 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109ff8c-0f81-4af1-b4d9-b2907f79062c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:09.902087 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109ff8c-0f81-4af1-b4d9-b2907f79062c None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:09.902411 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109ff8c-0f81-4af1-b4d9-b2907f79062c None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:09.902687 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109ff8c-0f81-4af1-b4d9-b2907f79062c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:09.913770 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1109ff8c-0f81-4af1-b4d9-b2907f79062c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:09.915321 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1109ff8c-0f81-4af1-b4d9-b2907f79062c None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:09.931100 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-1109ff8c-0f81-4af1-b4d9-b2907f79062c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:34:09.931100 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:34:09.931100 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:34:09.931100 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:34:09.931100 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:34:09.931100 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:34:09.931100 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:34:09.931100 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:34:09.931100 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:34:09.931100 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:34:09.931100 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:34:09.931100 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:34:09.931100 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:34:09.931100 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:34:09.931100 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:34:09.931100 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:34:09.932286 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:34:09.932286 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:34:09.932286 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:34:09.932286 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:34:09.932286 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:34:09.932286 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:34:09.932286 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:34:09.932286 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:34:09.932286 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:34:09.932286 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:34:09.932286 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:34:09.932286 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:34:09.932286 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:34:09.932286 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:34:09.932286 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:34:09.932286 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:34:09.932286 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:34:09.938516 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:34:09.938516 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:34:09.938516 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:34:09.938516 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 621/1251] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:34:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:34:09.938761 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5293ea88-5e50-4e99-b5b2-85bc9243c291 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:10.052857 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5293ea88-5e50-4e99-b5b2-85bc9243c291 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:10.088645 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d895278a-d42d-4152-89dc-9e82b151cd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:10.089249 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895278a-d42d-4152-89dc-9e82b151cd2a None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:10.089528 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895278a-d42d-4152-89dc-9e82b151cd2a None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:10.089802 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895278a-d42d-4152-89dc-9e82b151cd2a None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:10.090438 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d895278a-d42d-4152-89dc-9e82b151cd2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:34:10.090742 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d895278a-d42d-4152-89dc-9e82b151cd2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:10.092143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d895278a-d42d-4152-89dc-9e82b151cd2a None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:10.104093 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 631/1252] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:34:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:34:10.109942 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109ff8c-0f81-4af1-b4d9-b2907f79062c None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:10.230261 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109ff8c-0f81-4af1-b4d9-b2907f79062c None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:10.259560 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:10.268386 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd18f3-226e-4fd4-9199-a888fc170016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:10.269031 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd18f3-226e-4fd4-9199-a888fc170016 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:10.269660 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd18f3-226e-4fd4-9199-a888fc170016 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:10.269932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd18f3-226e-4fd4-9199-a888fc170016 None admin] PATH_INFO: `/v3/users/placement` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:10.280392 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcd18f3-226e-4fd4-9199-a888fc170016 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:10.281852 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcd18f3-226e-4fd4-9199-a888fc170016 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:10.297804 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-7dcd18f3-226e-4fd4-9199-a888fc170016 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 17 23:34:10.297804 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:34:10.297804 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:34:10.297804 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:34:10.297804 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:34:10.297804 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:34:10.297804 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:34:10.297804 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:34:10.297804 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:34:10.297804 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:34:10.297804 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:34:10.297804 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:34:10.297804 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 23:34:10.297804 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 23:34:10.297804 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 23:34:10.297804 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 23:34:10.298820 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:34:10.298820 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:34:10.298820 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 23:34:10.298820 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:34:10.298820 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 23:34:10.298820 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:34:10.298820 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:34:10.298820 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:34:10.298820 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:34:10.298820 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:34:10.298820 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:34:10.298820 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:34:10.298820 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:34:10.298820 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:34:10.298820 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:34:10.298820 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:34:10.298820 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:34:10.298820 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:34:10.301577 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:34:10.301577 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:34:10.301577 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 23:34:10.301577 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 23:34:10.301577 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 23:34:10.301577 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 23:34:10.301577 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 23:34:10.301577 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 23:34:10.301577 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 17 23:34:10.301577 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:34:10.301577 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 622/1253] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:34:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:34:10.305684 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d895278a-d42d-4152-89dc-9e82b151cd2a None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:10.370527 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:10.423204 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d895278a-d42d-4152-89dc-9e82b151cd2a None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:10.471042 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a6e14-ba49-42ea-82c7-373bf05a9013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:10.472299 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a6e14-ba49-42ea-82c7-373bf05a9013 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:10.473535 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a6e14-ba49-42ea-82c7-373bf05a9013 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:10.474174 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a6e14-ba49-42ea-82c7-373bf05a9013 None admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:10.475415 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935a6e14-ba49-42ea-82c7-373bf05a9013 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:34:10.476012 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935a6e14-ba49-42ea-82c7-373bf05a9013 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:10.477899 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935a6e14-ba49-42ea-82c7-373bf05a9013 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:10.499223 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 632/1254] 10.222.0.26 () {56 vars in 1143 bytes} [Wed Aug 17 23:34:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:34:10.505602 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd18f3-226e-4fd4-9199-a888fc170016 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:10.618964 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd18f3-226e-4fd4-9199-a888fc170016 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:10.662043 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eca629-9bbe-4889-91c0-a1fcc78038a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:10.662886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eca629-9bbe-4889-91c0-a1fcc78038a5 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:10.663203 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eca629-9bbe-4889-91c0-a1fcc78038a5 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:10.663630 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eca629-9bbe-4889-91c0-a1fcc78038a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:10.732032 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:10.749081 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eca629-9bbe-4889-91c0-a1fcc78038a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:10.750784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eca629-9bbe-4889-91c0-a1fcc78038a5 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:10.869941 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 623/1255] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:34:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 23:34:10.876336 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a6e14-ba49-42ea-82c7-373bf05a9013 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:10.983674 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a6e14-ba49-42ea-82c7-373bf05a9013 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:10.999381 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:11.023939 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93174840-1270-48bc-b2c1-36bc742de4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:11.024533 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93174840-1270-48bc-b2c1-36bc742de4c1 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:11.024810 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93174840-1270-48bc-b2c1-36bc742de4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:11.025080 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93174840-1270-48bc-b2c1-36bc742de4c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:11.036797 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93174840-1270-48bc-b2c1-36bc742de4c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:11.038273 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93174840-1270-48bc-b2c1-36bc742de4c1 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:11.053812 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-93174840-1270-48bc-b2c1-36bc742de4c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:34:11.053812 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:34:11.053812 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:34:11.053812 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:34:11.053812 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:34:11.053812 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:34:11.053812 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:34:11.053812 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:34:11.053812 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:34:11.053812 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:34:11.053812 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:34:11.053812 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:34:11.053812 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:34:11.053812 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:34:11.053812 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:34:11.053812 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:34:11.055515 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:34:11.055515 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:34:11.055515 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:34:11.055515 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:34:11.055515 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:34:11.055515 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:34:11.055515 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:34:11.055515 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:34:11.055515 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:34:11.055515 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:34:11.055515 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:34:11.055515 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:34:11.055515 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:34:11.055515 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:34:11.055515 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:34:11.055515 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:34:11.055515 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:34:11.056825 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:34:11.056825 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:34:11.056825 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:34:11.056825 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 633/1256] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:34:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:34:11.060942 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eca629-9bbe-4889-91c0-a1fcc78038a5 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:11.152016 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:11.166035 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eca629-9bbe-4889-91c0-a1fcc78038a5 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:11.201846 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76c5cc-371a-4ab6-91ec-4e54e7dab0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:11.202456 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76c5cc-371a-4ab6-91ec-4e54e7dab0bd None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:11.202735 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76c5cc-371a-4ab6-91ec-4e54e7dab0bd None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:11.203006 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76c5cc-371a-4ab6-91ec-4e54e7dab0bd None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:11.203655 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76c5cc-371a-4ab6-91ec-4e54e7dab0bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:34:11.203987 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76c5cc-371a-4ab6-91ec-4e54e7dab0bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:11.205367 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76c5cc-371a-4ab6-91ec-4e54e7dab0bd None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:11.217198 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 624/1257] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:34:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:34:11.223032 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93174840-1270-48bc-b2c1-36bc742de4c1 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:11.328866 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93174840-1270-48bc-b2c1-36bc742de4c1 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:11.364118 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa68d61-feac-4c8f-b8f8-df2bf8eb62b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:11.364759 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa68d61-feac-4c8f-b8f8-df2bf8eb62b0 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:11.365037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa68d61-feac-4c8f-b8f8-df2bf8eb62b0 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:11.365305 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa68d61-feac-4c8f-b8f8-df2bf8eb62b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:11.372597 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa68d61-feac-4c8f-b8f8-df2bf8eb62b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:11.374081 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa68d61-feac-4c8f-b8f8-df2bf8eb62b0 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:11.385608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-9fa68d61-feac-4c8f-b8f8-df2bf8eb62b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:34:11.385608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:34:11.385608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:34:11.385608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:34:11.385608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:34:11.385608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:34:11.385608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:34:11.385608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:34:11.385608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:34:11.385608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:34:11.385608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:34:11.385608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:34:11.385608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 23:34:11.385608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 23:34:11.385608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 23:34:11.386562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 23:34:11.386562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:34:11.386562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:34:11.386562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:34:11.386562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:34:11.386562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:34:11.386562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:34:11.386562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:34:11.386562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:34:11.386562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:34:11.386562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:34:11.386562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:34:11.386562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:34:11.386562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:34:11.386562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:34:11.386562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:34:11.386562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:34:11.387481 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 23:34:11.387481 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 23:34:11.387481 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 23:34:11.387481 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 23:34:11.387481 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 23:34:11.387481 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 23:34:11.387481 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:34:11.387481 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:34:11.387481 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 634/1258] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:34:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:34:11.392767 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76c5cc-371a-4ab6-91ec-4e54e7dab0bd None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:11.494831 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76c5cc-371a-4ab6-91ec-4e54e7dab0bd None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:11.529648 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce2e8b-e843-43d7-bed8-86910025d103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:11.530258 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce2e8b-e843-43d7-bed8-86910025d103 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:11.530536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce2e8b-e843-43d7-bed8-86910025d103 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:11.530807 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce2e8b-e843-43d7-bed8-86910025d103 None admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:11.531497 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce2e8b-e843-43d7-bed8-86910025d103 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:34:11.531803 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce2e8b-e843-43d7-bed8-86910025d103 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:11.533262 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce2e8b-e843-43d7-bed8-86910025d103 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:11.546570 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 625/1259] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 17 23:34:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:34:11.553074 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa68d61-feac-4c8f-b8f8-df2bf8eb62b0 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:11.579183 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:11.655477 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa68d61-feac-4c8f-b8f8-df2bf8eb62b0 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:11.687215 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81047ce-24e0-439b-9cc4-9e3833600002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:11.687810 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81047ce-24e0-439b-9cc4-9e3833600002 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:11.688120 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81047ce-24e0-439b-9cc4-9e3833600002 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:11.688393 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81047ce-24e0-439b-9cc4-9e3833600002 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:11.689105 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81047ce-24e0-439b-9cc4-9e3833600002 None admin] RBAC: Adding query filter params (role.id=33bd60d3b0e94dcbbfe97e87f91a8d78, scope.project.id=4de86915ef4340248ba8a8f51d56594e, user.id=1fc57d911a56423e96c736bc96270ba8) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:34:11.689411 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81047ce-24e0-439b-9cc4-9e3833600002 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:11.690824 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81047ce-24e0-439b-9cc4-9e3833600002 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:11.696533 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 635/1260] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:34:11 2022] GET /identity/v3/role_assignments?user.id=1fc57d911a56423e96c736bc96270ba8&scope.project.id=4de86915ef4340248ba8a8f51d56594e&role.id=33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:34:13.545405 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c3ce35-6d64-4cab-9975-93010ba3cc4e None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:13.545915 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c3ce35-6d64-4cab-9975-93010ba3cc4e None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:13.546015 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c3ce35-6d64-4cab-9975-93010ba3cc4e None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:13.547285 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 626/1261] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:34:13.552367 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a94b2-2892-4055-8804-26b6ad5e5bdd None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:13.552772 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a94b2-2892-4055-8804-26b6ad5e5bdd None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:13.552917 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a94b2-2892-4055-8804-26b6ad5e5bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:13.570462 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-968a94b2-2892-4055-8804-26b6ad5e5bdd None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:34:13.571645 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-968a94b2-2892-4055-8804-26b6ad5e5bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:34:13.636364 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 636/1262] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:34:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:34:13.900822 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c3ce35-6d64-4cab-9975-93010ba3cc4e None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:13.912131 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:13.999559 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c3ce35-6d64-4cab-9975-93010ba3cc4e None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:14.032526 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b114f-79f2-4b3e-83b5-186a6164970c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:14.033134 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b114f-79f2-4b3e-83b5-186a6164970c None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:14.033408 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b114f-79f2-4b3e-83b5-186a6164970c None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:14.033675 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b114f-79f2-4b3e-83b5-186a6164970c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:14.043270 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b114f-79f2-4b3e-83b5-186a6164970c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:34:14.047851 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-915b114f-79f2-4b3e-83b5-186a6164970c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:34:14.047851 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:34:14.047851 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:34:14.047851 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:34:14.047851 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:34:14.047851 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:34:14.047851 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:34:14.047851 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:34:14.047851 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:34:14.047851 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:34:14.047851 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:34:14.047851 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:34:14.047851 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:34:14.047851 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:34:14.047851 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:34:14.047851 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:34:14.048960 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:34:14.048960 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:34:14.048960 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:34:14.048960 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:34:14.048960 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:34:14.048960 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:34:14.048960 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:34:14.048960 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:34:14.048960 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:34:14.048960 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:34:14.048960 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:34:14.048960 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:34:14.048960 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:34:14.048960 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:34:14.048960 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:34:14.048960 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:34:14.049985 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:34:14.049985 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:34:14.049985 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:34:14.049985 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:34:14.049985 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:34:14.049985 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:34:14.049985 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:34:14.049985 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:34:14.049985 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:34:14.049985 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:34:14.049985 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:34:14.049985 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:34:14.049985 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 627/1263] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 17 23:34:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:34:14.056914 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a94b2-2892-4055-8804-26b6ad5e5bdd None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:14.075201 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:14.153960 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a94b2-2892-4055-8804-26b6ad5e5bdd None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:14.186957 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb5af5-296e-4bf1-aae8-e1831ecea59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:14.187552 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb5af5-296e-4bf1-aae8-e1831ecea59a None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:14.187829 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb5af5-296e-4bf1-aae8-e1831ecea59a None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:14.188132 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb5af5-296e-4bf1-aae8-e1831ecea59a None admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:14.188793 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb5af5-296e-4bf1-aae8-e1831ecea59a None admin] RBAC: Adding query filter params (name=admin) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:34:14.189092 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb5af5-296e-4bf1-aae8-e1831ecea59a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:14.190447 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb5af5-296e-4bf1-aae8-e1831ecea59a None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:14.198672 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 637/1264] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:34:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:34:14.204294 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b114f-79f2-4b3e-83b5-186a6164970c None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:14.268797 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:14.301053 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b114f-79f2-4b3e-83b5-186a6164970c None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:14.333671 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42881600-cac7-4961-b527-634ed8d786d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:14.334292 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42881600-cac7-4961-b527-634ed8d786d0 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:14.334573 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42881600-cac7-4961-b527-634ed8d786d0 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:14.334847 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42881600-cac7-4961-b527-634ed8d786d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:14.344505 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42881600-cac7-4961-b527-634ed8d786d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:14.345949 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42881600-cac7-4961-b527-634ed8d786d0 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:14.359897 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-42881600-cac7-4961-b527-634ed8d786d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:34:14.359897 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:34:14.359897 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:34:14.359897 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:34:14.359897 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:34:14.359897 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:34:14.359897 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:34:14.359897 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:34:14.359897 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:34:14.359897 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:34:14.359897 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:34:14.359897 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:34:14.359897 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:34:14.359897 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:34:14.359897 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:34:14.359897 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:34:14.361007 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:34:14.361007 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:34:14.361007 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:34:14.361007 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:34:14.361007 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:34:14.361007 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:34:14.361007 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:34:14.361007 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:34:14.361007 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:34:14.361007 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:34:14.361007 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:34:14.361007 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:34:14.361007 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:34:14.361007 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:34:14.361007 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:34:14.361007 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:34:14.361007 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:34:14.362127 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:34:14.362127 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:34:14.362127 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:34:14.362127 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 628/1265] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:34:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:34:14.367318 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb5af5-296e-4bf1-aae8-e1831ecea59a None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:14.446124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:14.465160 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb5af5-296e-4bf1-aae8-e1831ecea59a None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:14.497792 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3aa1e9-d88a-4801-b5bc-e9c198cd1a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:14.498386 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3aa1e9-d88a-4801-b5bc-e9c198cd1a17 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:14.498662 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3aa1e9-d88a-4801-b5bc-e9c198cd1a17 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:14.498930 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3aa1e9-d88a-4801-b5bc-e9c198cd1a17 None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:14.499553 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3aa1e9-d88a-4801-b5bc-e9c198cd1a17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:34:14.499849 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3aa1e9-d88a-4801-b5bc-e9c198cd1a17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:14.501240 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3aa1e9-d88a-4801-b5bc-e9c198cd1a17 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:14.512608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 638/1266] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:34:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:34:14.518239 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42881600-cac7-4961-b527-634ed8d786d0 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:14.615299 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42881600-cac7-4961-b527-634ed8d786d0 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:14.633964 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:14.648247 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43c2b3-c332-4899-a0c0-78569784d0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:14.648848 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43c2b3-c332-4899-a0c0-78569784d0d6 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:14.649126 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43c2b3-c332-4899-a0c0-78569784d0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:14.649398 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43c2b3-c332-4899-a0c0-78569784d0d6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:14.656790 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b43c2b3-c332-4899-a0c0-78569784d0d6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:14.658228 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b43c2b3-c332-4899-a0c0-78569784d0d6 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:14.669461 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-9b43c2b3-c332-4899-a0c0-78569784d0d6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 17 23:34:14.669461 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:34:14.669461 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:34:14.669461 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:34:14.669461 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:34:14.669461 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:34:14.669461 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:34:14.669461 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:34:14.669461 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:34:14.669461 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:34:14.669461 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:34:14.669461 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:34:14.669461 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 23:34:14.669461 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 23:34:14.669461 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 23:34:14.669461 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 23:34:14.670542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:34:14.670542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:34:14.670542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 23:34:14.670542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:34:14.670542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 23:34:14.670542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:34:14.670542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:34:14.670542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:34:14.670542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:34:14.670542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:34:14.670542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:34:14.670542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:34:14.670542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:34:14.670542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:34:14.670542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:34:14.670542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:34:14.670542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:34:14.670542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:34:14.671582 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:34:14.671582 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:34:14.671582 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 23:34:14.671582 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 23:34:14.671582 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 23:34:14.671582 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 23:34:14.671582 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 23:34:14.671582 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 23:34:14.671582 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 17 23:34:14.671582 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:34:14.671582 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 629/1267] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:34:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:34:14.676814 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3aa1e9-d88a-4801-b5bc-e9c198cd1a17 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:14.781565 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3aa1e9-d88a-4801-b5bc-e9c198cd1a17 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:14.814944 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9270aa9-a659-41c5-aa5a-868b1b661ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:14.815534 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9270aa9-a659-41c5-aa5a-868b1b661ce9 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:14.815813 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9270aa9-a659-41c5-aa5a-868b1b661ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:14.816115 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9270aa9-a659-41c5-aa5a-868b1b661ce9 None admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:14.816831 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9270aa9-a659-41c5-aa5a-868b1b661ce9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:34:14.817132 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9270aa9-a659-41c5-aa5a-868b1b661ce9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:14.818577 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9270aa9-a659-41c5-aa5a-868b1b661ce9 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:14.826850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:14.835116 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 639/1268] 10.222.0.26 () {56 vars in 1143 bytes} [Wed Aug 17 23:34:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:34:14.840753 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43c2b3-c332-4899-a0c0-78569784d0d6 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:14.937745 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43c2b3-c332-4899-a0c0-78569784d0d6 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:14.969742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3422aac0-b4dd-46a9-b2c6-034ab2fdcd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:14.970356 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422aac0-b4dd-46a9-b2c6-034ab2fdcd36 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:14.970634 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422aac0-b4dd-46a9-b2c6-034ab2fdcd36 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:14.970903 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422aac0-b4dd-46a9-b2c6-034ab2fdcd36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:14.981543 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3422aac0-b4dd-46a9-b2c6-034ab2fdcd36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:14.982978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3422aac0-b4dd-46a9-b2c6-034ab2fdcd36 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:14.996537 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-3422aac0-b4dd-46a9-b2c6-034ab2fdcd36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:34:14.996537 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:34:14.996537 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:34:14.996537 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:34:14.996537 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:34:14.996537 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:34:14.996537 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:34:14.996537 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:34:14.996537 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:34:14.996537 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:34:14.996537 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:34:14.996537 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:34:14.996537 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:34:14.996537 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:34:14.996537 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:34:14.996537 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:34:14.997619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:34:14.997619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:34:14.997619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:34:14.997619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:34:14.997619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:34:14.997619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:34:14.997619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:34:14.997619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:34:14.997619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:34:14.997619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:34:14.997619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:34:14.997619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:34:14.997619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:34:14.997619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:34:14.997619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:34:14.997619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:34:14.997619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:34:14.998542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:34:14.998542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:34:14.998542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:34:14.998542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 630/1269] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:34:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:34:15.003296 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9270aa9-a659-41c5-aa5a-868b1b661ce9 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:15.098064 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9270aa9-a659-41c5-aa5a-868b1b661ce9 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:15.129459 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd56aaa-a034-4ea0-90b8-18d29934c78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:15.130040 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd56aaa-a034-4ea0-90b8-18d29934c78c None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:15.130315 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd56aaa-a034-4ea0-90b8-18d29934c78c None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:15.130584 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd56aaa-a034-4ea0-90b8-18d29934c78c None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:15.131209 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd56aaa-a034-4ea0-90b8-18d29934c78c None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:34:15.131504 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd56aaa-a034-4ea0-90b8-18d29934c78c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:15.132871 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd56aaa-a034-4ea0-90b8-18d29934c78c None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:15.143414 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 640/1270] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:34:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:34:15.149383 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3422aac0-b4dd-46a9-b2c6-034ab2fdcd36 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:15.181271 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:15.248928 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3422aac0-b4dd-46a9-b2c6-034ab2fdcd36 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:15.281966 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769a920-a166-4e31-b43b-facbee4974e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:15.282575 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769a920-a166-4e31-b43b-facbee4974e5 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:15.282856 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769a920-a166-4e31-b43b-facbee4974e5 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:15.283128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769a920-a166-4e31-b43b-facbee4974e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:15.289559 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8769a920-a166-4e31-b43b-facbee4974e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:15.290991 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8769a920-a166-4e31-b43b-facbee4974e5 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:15.301858 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-8769a920-a166-4e31-b43b-facbee4974e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:34:15.301858 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:34:15.301858 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:34:15.301858 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:34:15.301858 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:34:15.301858 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:34:15.301858 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:34:15.301858 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:34:15.301858 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:34:15.301858 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:34:15.301858 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:34:15.301858 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:34:15.301858 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 23:34:15.301858 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 23:34:15.301858 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 23:34:15.302881 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 23:34:15.302881 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:34:15.302881 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:34:15.302881 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:34:15.302881 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:34:15.302881 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:34:15.302881 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:34:15.302881 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:34:15.302881 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:34:15.302881 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:34:15.302881 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:34:15.302881 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:34:15.302881 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:34:15.302881 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:34:15.302881 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:34:15.302881 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:34:15.302881 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:34:15.304220 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 23:34:15.304220 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 23:34:15.304220 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 23:34:15.304220 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 23:34:15.304220 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 23:34:15.304220 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 23:34:15.304220 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:34:15.304220 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:34:15.304220 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 631/1271] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:34:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:34:15.308692 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd56aaa-a034-4ea0-90b8-18d29934c78c None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:15.364396 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:15.405522 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd56aaa-a034-4ea0-90b8-18d29934c78c None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:15.436970 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0c2b9f-f51b-4e0a-a5f6-9ebceac3661a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:15.437556 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c2b9f-f51b-4e0a-a5f6-9ebceac3661a None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:15.437831 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c2b9f-f51b-4e0a-a5f6-9ebceac3661a None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:15.438100 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c2b9f-f51b-4e0a-a5f6-9ebceac3661a None admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:15.438769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0c2b9f-f51b-4e0a-a5f6-9ebceac3661a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:34:15.439067 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0c2b9f-f51b-4e0a-a5f6-9ebceac3661a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:15.440526 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0c2b9f-f51b-4e0a-a5f6-9ebceac3661a None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:15.452738 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 641/1272] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 17 23:34:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:34:15.459230 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769a920-a166-4e31-b43b-facbee4974e5 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:15.546833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:15.555563 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769a920-a166-4e31-b43b-facbee4974e5 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:15.588181 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef6a86-9895-4ebc-b19a-6384a18a3ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:15.588806 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef6a86-9895-4ebc-b19a-6384a18a3ba1 None admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:15.589083 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef6a86-9895-4ebc-b19a-6384a18a3ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:15.589351 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef6a86-9895-4ebc-b19a-6384a18a3ba1 None admin] PATH_INFO: `/v3/projects/4de86915ef4340248ba8a8f51d56594e/users/1fc57d911a56423e96c736bc96270ba8/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:15.591207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef6a86-9895-4ebc-b19a-6384a18a3ba1 None admin] RBAC: Authorizing `identity:create_grant(project_id=4de86915ef4340248ba8a8f51d56594e, user_id=1fc57d911a56423e96c736bc96270ba8, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:15.592667 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef6a86-9895-4ebc-b19a-6384a18a3ba1 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:15.604452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 632/1273] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Aug 17 23:34:15 2022] PUT /identity/v3/projects/4de86915ef4340248ba8a8f51d56594e/users/1fc57d911a56423e96c736bc96270ba8/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:34:17.466036 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d80fe7-40bd-4db8-96eb-8b535c714f90 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:17.466470 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d80fe7-40bd-4db8-96eb-8b535c714f90 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:17.466599 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d80fe7-40bd-4db8-96eb-8b535c714f90 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:17.467903 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 642/1274] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:34:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:34:17.473715 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c936e-c462-4c60-8bc2-dc4507c9dbde None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:17.474087 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c936e-c462-4c60-8bc2-dc4507c9dbde None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:17.474259 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c936e-c462-4c60-8bc2-dc4507c9dbde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:17.492034 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-551c936e-c462-4c60-8bc2-dc4507c9dbde None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:34:17.493217 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-551c936e-c462-4c60-8bc2-dc4507c9dbde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:34:17.557514 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 633/1275] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:34:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:34:17.823465 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d80fe7-40bd-4db8-96eb-8b535c714f90 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:17.922184 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d80fe7-40bd-4db8-96eb-8b535c714f90 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:17.938866 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:17.955878 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ce937-fd01-4d0d-bfee-9b3dc2c18f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:17.956520 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ce937-fd01-4d0d-bfee-9b3dc2c18f44 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:17.956795 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ce937-fd01-4d0d-bfee-9b3dc2c18f44 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:17.957063 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ce937-fd01-4d0d-bfee-9b3dc2c18f44 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:17.966647 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880ce937-fd01-4d0d-bfee-9b3dc2c18f44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 23:34:17.971287 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-880ce937-fd01-4d0d-bfee-9b3dc2c18f44 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:34:17.971287 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:34:17.971287 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:34:17.971287 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:34:17.971287 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:34:17.971287 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:34:17.971287 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:34:17.971287 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:34:17.971287 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:34:17.971287 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:34:17.971287 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:34:17.971287 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:34:17.971287 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 23:34:17.971287 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 23:34:17.971287 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 23:34:17.971287 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 23:34:17.972391 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 23:34:17.972391 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 23:34:17.972391 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 23:34:17.972391 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 23:34:17.972391 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:34:17.972391 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:34:17.972391 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:34:17.972391 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:34:17.972391 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:34:17.972391 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:34:17.972391 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:34:17.972391 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:34:17.972391 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:34:17.972391 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:34:17.972391 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:34:17.972391 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:34:17.973451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:34:17.973451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:34:17.973451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:34:17.973451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:34:17.973451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 23:34:17.973451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 23:34:17.973451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 23:34:17.973451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 23:34:17.973451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 23:34:17.973451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 23:34:17.973451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 23:34:17.973451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:34:17.973451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 643/1276] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Aug 17 23:34:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:34:17.978277 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c936e-c462-4c60-8bc2-dc4507c9dbde None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:18.056677 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:18.075499 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c936e-c462-4c60-8bc2-dc4507c9dbde None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:18.107488 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c4d9bc-79b4-44d7-ae4f-3328062216c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:18.108110 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c4d9bc-79b4-44d7-ae4f-3328062216c7 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:18.108390 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c4d9bc-79b4-44d7-ae4f-3328062216c7 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:18.108659 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c4d9bc-79b4-44d7-ae4f-3328062216c7 None admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:18.109320 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c4d9bc-79b4-44d7-ae4f-3328062216c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:34:18.109620 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c4d9bc-79b4-44d7-ae4f-3328062216c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:18.110987 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c4d9bc-79b4-44d7-ae4f-3328062216c7 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:18.119212 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 634/1277] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Aug 17 23:34:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:34:18.124795 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ce937-fd01-4d0d-bfee-9b3dc2c18f44 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:18.220936 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ce937-fd01-4d0d-bfee-9b3dc2c18f44 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:18.253152 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017bf629-5dcd-4b4b-99b3-e81a992499f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:18.253746 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017bf629-5dcd-4b4b-99b3-e81a992499f4 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:18.254038 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017bf629-5dcd-4b4b-99b3-e81a992499f4 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:18.254304 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017bf629-5dcd-4b4b-99b3-e81a992499f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:18.264610 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017bf629-5dcd-4b4b-99b3-e81a992499f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:18.266045 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017bf629-5dcd-4b4b-99b3-e81a992499f4 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:18.279728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-017bf629-5dcd-4b4b-99b3-e81a992499f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:34:18.279728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:34:18.279728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:34:18.279728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:34:18.279728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:34:18.279728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:34:18.279728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:34:18.279728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:34:18.279728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:34:18.279728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:34:18.279728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:34:18.279728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:34:18.279728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:34:18.279728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:34:18.279728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:34:18.279728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:34:18.280922 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:34:18.280922 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:34:18.280922 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:34:18.280922 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:34:18.280922 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:34:18.280922 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:34:18.280922 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:34:18.280922 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:34:18.280922 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:34:18.280922 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:34:18.280922 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:34:18.280922 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:34:18.280922 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:34:18.280922 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:34:18.280922 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:34:18.280922 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:34:18.280922 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:34:18.282244 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:34:18.282244 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:34:18.282244 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:34:18.282244 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 644/1278] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:34:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:34:18.287053 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c4d9bc-79b4-44d7-ae4f-3328062216c7 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:18.383259 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c4d9bc-79b4-44d7-ae4f-3328062216c7 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:18.415444 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee01cc9-62a3-4147-808f-92b2f286d866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:18.416065 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee01cc9-62a3-4147-808f-92b2f286d866 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:18.416347 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee01cc9-62a3-4147-808f-92b2f286d866 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:18.416618 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee01cc9-62a3-4147-808f-92b2f286d866 None admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:18.417311 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee01cc9-62a3-4147-808f-92b2f286d866 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:34:18.417612 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee01cc9-62a3-4147-808f-92b2f286d866 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:18.418976 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee01cc9-62a3-4147-808f-92b2f286d866 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:18.430780 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 635/1279] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:34:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:34:18.436280 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017bf629-5dcd-4b4b-99b3-e81a992499f4 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:18.466264 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:18.533780 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017bf629-5dcd-4b4b-99b3-e81a992499f4 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:18.566672 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400a4908-4150-474c-acff-1d85c03decc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:18.567321 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a4908-4150-474c-acff-1d85c03decc0 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:18.567600 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a4908-4150-474c-acff-1d85c03decc0 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:18.567872 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a4908-4150-474c-acff-1d85c03decc0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:18.574992 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400a4908-4150-474c-acff-1d85c03decc0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:18.576459 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400a4908-4150-474c-acff-1d85c03decc0 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:18.587671 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-400a4908-4150-474c-acff-1d85c03decc0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 17 23:34:18.587671 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:34:18.587671 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:34:18.587671 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:34:18.587671 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:34:18.587671 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:34:18.587671 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:34:18.587671 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:34:18.587671 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:34:18.587671 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:34:18.587671 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:34:18.587671 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:34:18.587671 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 23:34:18.587671 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 23:34:18.587671 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 23:34:18.587671 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 23:34:18.588819 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:34:18.588819 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:34:18.588819 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 23:34:18.588819 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:34:18.588819 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 23:34:18.588819 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 23:34:18.588819 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:34:18.588819 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:34:18.588819 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:34:18.588819 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:34:18.588819 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:34:18.588819 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:34:18.588819 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:34:18.588819 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:34:18.588819 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:34:18.588819 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:34:18.588819 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:34:18.588819 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:34:18.589815 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:34:18.589815 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:34:18.589815 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 23:34:18.589815 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 23:34:18.589815 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 23:34:18.589815 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 23:34:18.589815 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 23:34:18.589815 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 23:34:18.589815 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 17 23:34:18.589815 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:34:18.589815 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 645/1280] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:34:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:34:18.594509 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee01cc9-62a3-4147-808f-92b2f286d866 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:18.598752 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:18.690764 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee01cc9-62a3-4147-808f-92b2f286d866 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:18.723610 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6e09b-a8cb-4d1f-bf44-4ca26728e74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:18.724257 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6e09b-a8cb-4d1f-bf44-4ca26728e74e None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:18.724535 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6e09b-a8cb-4d1f-bf44-4ca26728e74e None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:18.724802 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6e09b-a8cb-4d1f-bf44-4ca26728e74e None admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:18.725551 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe6e09b-a8cb-4d1f-bf44-4ca26728e74e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:34:18.725853 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe6e09b-a8cb-4d1f-bf44-4ca26728e74e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:18.727281 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe6e09b-a8cb-4d1f-bf44-4ca26728e74e None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:18.743545 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 636/1281] 10.222.0.26 () {56 vars in 1143 bytes} [Wed Aug 17 23:34:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:34:18.749388 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400a4908-4150-474c-acff-1d85c03decc0 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:18.837086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:18.848804 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400a4908-4150-474c-acff-1d85c03decc0 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:18.880451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd16fc74-a967-4014-b578-78f1b19424dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:18.881040 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16fc74-a967-4014-b578-78f1b19424dc None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:18.881316 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16fc74-a967-4014-b578-78f1b19424dc None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:18.881585 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16fc74-a967-4014-b578-78f1b19424dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:18.945175 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd16fc74-a967-4014-b578-78f1b19424dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:18.946547 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd16fc74-a967-4014-b578-78f1b19424dc None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:19.031941 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:19.042371 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 646/1282] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Aug 17 23:34:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 23:34:19.048437 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6e09b-a8cb-4d1f-bf44-4ca26728e74e None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:19.107031 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:19.143818 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6e09b-a8cb-4d1f-bf44-4ca26728e74e None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:19.176460 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac2bd29-da7e-4342-ac42-b45c5294486d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:19.177078 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2bd29-da7e-4342-ac42-b45c5294486d None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:19.177385 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2bd29-da7e-4342-ac42-b45c5294486d None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:19.177658 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2bd29-da7e-4342-ac42-b45c5294486d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:19.187131 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac2bd29-da7e-4342-ac42-b45c5294486d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:19.188606 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac2bd29-da7e-4342-ac42-b45c5294486d None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:19.202999 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-cac2bd29-da7e-4342-ac42-b45c5294486d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:34:19.202999 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:34:19.202999 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:34:19.202999 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:34:19.202999 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:34:19.202999 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:34:19.202999 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:34:19.202999 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:34:19.202999 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:34:19.202999 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:34:19.202999 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:34:19.202999 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:34:19.202999 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 23:34:19.202999 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 23:34:19.202999 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 23:34:19.202999 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 23:34:19.204338 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:34:19.204338 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:34:19.204338 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:34:19.204338 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:34:19.204338 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:34:19.204338 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:34:19.204338 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:34:19.204338 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:34:19.204338 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:34:19.204338 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:34:19.204338 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:34:19.204338 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:34:19.204338 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:34:19.204338 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:34:19.204338 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:34:19.204338 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:34:19.204338 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 23:34:19.205602 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 23:34:19.205602 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 23:34:19.205602 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:34:19.205602 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 637/1283] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:34:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:34:19.210222 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd16fc74-a967-4014-b578-78f1b19424dc None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:19.304762 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd16fc74-a967-4014-b578-78f1b19424dc None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:19.337745 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7b944-6b0d-40a3-a3a9-9a09b37cbaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:19.338334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7b944-6b0d-40a3-a3a9-9a09b37cbaf9 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:19.338611 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7b944-6b0d-40a3-a3a9-9a09b37cbaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:19.338882 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7b944-6b0d-40a3-a3a9-9a09b37cbaf9 None admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:19.339515 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e7b944-6b0d-40a3-a3a9-9a09b37cbaf9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:34:19.339814 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e7b944-6b0d-40a3-a3a9-9a09b37cbaf9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:19.341190 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e7b944-6b0d-40a3-a3a9-9a09b37cbaf9 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:19.351881 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 647/1284] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Aug 17 23:34:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:34:19.357232 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac2bd29-da7e-4342-ac42-b45c5294486d None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:19.452572 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac2bd29-da7e-4342-ac42-b45c5294486d None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:19.465673 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:19.485421 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb781e-ec21-47cc-8f6d-b6fe36509ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:19.486037 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb781e-ec21-47cc-8f6d-b6fe36509ee8 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:19.486318 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb781e-ec21-47cc-8f6d-b6fe36509ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:19.486592 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb781e-ec21-47cc-8f6d-b6fe36509ee8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:19.493638 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb781e-ec21-47cc-8f6d-b6fe36509ee8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:19.495088 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb781e-ec21-47cc-8f6d-b6fe36509ee8 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:19.506781 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application [None req-dddb781e-ec21-47cc-8f6d-b6fe36509ee8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:34:19.506781 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:34:19.506781 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:34:19.506781 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:34:19.506781 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:34:19.506781 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:34:19.506781 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:34:19.506781 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:34:19.506781 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:34:19.506781 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:34:19.506781 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:34:19.506781 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:34:19.506781 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 23:34:19.506781 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 23:34:19.506781 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 23:34:19.507833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 23:34:19.507833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:34:19.507833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:34:19.507833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:34:19.507833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:34:19.507833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:34:19.507833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:34:19.507833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:34:19.507833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application with Lock( Aug 17 23:34:19.507833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:34:19.507833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:34:19.507833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:34:19.507833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:34:19.507833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:34:19.507833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:34:19.507833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:34:19.507833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:34:19.508827 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 23:34:19.508827 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 23:34:19.508827 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 23:34:19.508827 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 23:34:19.508827 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 23:34:19.508827 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 23:34:19.508827 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 23:34:19.508827 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: ERROR keystone.server.flask.application Aug 17 23:34:19.508827 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 638/1285] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Aug 17 23:34:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:34:19.513648 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7b944-6b0d-40a3-a3a9-9a09b37cbaf9 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:19.561934 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:19.610066 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7b944-6b0d-40a3-a3a9-9a09b37cbaf9 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:19.642802 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c026e-d8b9-4a41-b7a2-5b09ed53452b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:19.643405 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c026e-d8b9-4a41-b7a2-5b09ed53452b None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:19.643684 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c026e-d8b9-4a41-b7a2-5b09ed53452b None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:19.643955 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c026e-d8b9-4a41-b7a2-5b09ed53452b None admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:19.644656 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c026e-d8b9-4a41-b7a2-5b09ed53452b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:34:19.644962 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c026e-d8b9-4a41-b7a2-5b09ed53452b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:19.646427 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c026e-d8b9-4a41-b7a2-5b09ed53452b None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:19.658032 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 648/1286] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Aug 17 23:34:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:34:19.663932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb781e-ec21-47cc-8f6d-b6fe36509ee8 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:19.760393 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb781e-ec21-47cc-8f6d-b6fe36509ee8 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:19.792471 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bac7fc-4b2b-44a4-89f5-1676fc836ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:19.793049 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bac7fc-4b2b-44a4-89f5-1676fc836ce1 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:19.793328 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bac7fc-4b2b-44a4-89f5-1676fc836ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:19.793598 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bac7fc-4b2b-44a4-89f5-1676fc836ce1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:19.794309 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bac7fc-4b2b-44a4-89f5-1676fc836ce1 None admin] RBAC: Adding query filter params (role.id=33bd60d3b0e94dcbbfe97e87f91a8d78, scope.project.id=4de86915ef4340248ba8a8f51d56594e, user.id=1fc57d911a56423e96c736bc96270ba8) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:34:19.794614 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bac7fc-4b2b-44a4-89f5-1676fc836ce1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:19.796053 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bac7fc-4b2b-44a4-89f5-1676fc836ce1 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:19.802110 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 639/1287] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Aug 17 23:34:19 2022] GET /identity/v3/role_assignments?user.id=1fc57d911a56423e96c736bc96270ba8&scope.project.id=4de86915ef4340248ba8a8f51d56594e&role.id=33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:34:21.666175 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daed02a-0f93-436e-8586-da616249bdcb None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:21.666725 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daed02a-0f93-436e-8586-da616249bdcb None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:21.666725 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daed02a-0f93-436e-8586-da616249bdcb None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:21.668070 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 649/1288] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:34:21.673121 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd7b41-643e-4a51-ae2a-6482e6925ccc None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:21.673528 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd7b41-643e-4a51-ae2a-6482e6925ccc None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:21.673658 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd7b41-643e-4a51-ae2a-6482e6925ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:21.691069 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-8ecd7b41-643e-4a51-ae2a-6482e6925ccc None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:34:21.692277 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-8ecd7b41-643e-4a51-ae2a-6482e6925ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:34:21.720922 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:21.757948 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 640/1289] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:34:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:34:22.025779 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daed02a-0f93-436e-8586-da616249bdcb None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:22.125071 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daed02a-0f93-436e-8586-da616249bdcb None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:22.135671 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:22.159454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9dd9c2-c1bd-46b1-ac0c-5ea03fc03f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:22.160117 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9dd9c2-c1bd-46b1-ac0c-5ea03fc03f1a None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:22.160915 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9dd9c2-c1bd-46b1-ac0c-5ea03fc03f1a None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:22.161187 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9dd9c2-c1bd-46b1-ac0c-5ea03fc03f1a None admin] PATH_INFO: `/v3/services/placement` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:22.161767 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9dd9c2-c1bd-46b1-ac0c-5ea03fc03f1a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:22.163129 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9dd9c2-c1bd-46b1-ac0c-5ea03fc03f1a None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:22.172509 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-ca9dd9c2-c1bd-46b1-ac0c-5ea03fc03f1a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 17 23:34:22.172509 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:34:22.172509 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 23:34:22.172509 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 23:34:22.172509 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 23:34:22.172509 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 23:34:22.172509 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 23:34:22.172509 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:34:22.172509 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 17 23:34:22.172509 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:34:22.172509 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 23:34:22.172509 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:34:22.172509 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:34:22.172509 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:34:22.172509 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:34:22.172509 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:34:22.172509 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:34:22.173747 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:34:22.173747 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:34:22.173747 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:34:22.173747 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:34:22.173747 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:34:22.173747 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:34:22.173747 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 23:34:22.173747 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 23:34:22.173747 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 23:34:22.173747 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 23:34:22.173747 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:34:22.173747 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:34:22.173747 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:34:22.173747 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:34:22.173747 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:34:22.173747 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:34:22.174632 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:34:22.174632 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:34:22.174632 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:34:22.174632 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:34:22.174632 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:34:22.174632 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:34:22.174632 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:34:22.174632 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:34:22.174632 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:34:22.174632 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:34:22.174632 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 23:34:22.174632 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:34:22.174632 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 17 23:34:22.174632 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:34:22.174632 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 650/1290] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Aug 17 23:34:22 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:34:22.180194 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd7b41-643e-4a51-ae2a-6482e6925ccc None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:22.286929 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd7b41-643e-4a51-ae2a-6482e6925ccc None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:22.320572 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110337e0-1b49-4921-9a9b-f2726aa118e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:22.321167 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110337e0-1b49-4921-9a9b-f2726aa118e6 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:22.321443 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110337e0-1b49-4921-9a9b-f2726aa118e6 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:22.321713 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110337e0-1b49-4921-9a9b-f2726aa118e6 None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:22.323378 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110337e0-1b49-4921-9a9b-f2726aa118e6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:34:22.323692 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110337e0-1b49-4921-9a9b-f2726aa118e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:22.325090 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110337e0-1b49-4921-9a9b-f2726aa118e6 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:22.329797 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 641/1291] 10.222.0.26 () {56 vars in 1116 bytes} [Wed Aug 17 23:34:22 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:34:22.336048 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9dd9c2-c1bd-46b1-ac0c-5ea03fc03f1a None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:22.435662 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9dd9c2-c1bd-46b1-ac0c-5ea03fc03f1a None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:22.472023 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a042e-ca58-4e9a-a7ab-a892d039118f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:22.472634 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a042e-ca58-4e9a-a7ab-a892d039118f None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:22.473456 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a042e-ca58-4e9a-a7ab-a892d039118f None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:22.473740 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a042e-ca58-4e9a-a7ab-a892d039118f None admin] PATH_INFO: `/v3/services` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:22.474367 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5a042e-ca58-4e9a-a7ab-a892d039118f None admin] RBAC: Adding query filter params (type=placement) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:34:22.474671 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5a042e-ca58-4e9a-a7ab-a892d039118f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:22.476068 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5a042e-ca58-4e9a-a7ab-a892d039118f None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:22.480493 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 651/1292] 10.222.0.26 () {56 vars in 1116 bytes} [Wed Aug 17 23:34:22 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:34:24.347012 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633268fc-c89c-4e83-8001-34e893e7cfc0 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:24.347578 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633268fc-c89c-4e83-8001-34e893e7cfc0 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:24.347578 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633268fc-c89c-4e83-8001-34e893e7cfc0 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:24.349401 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 642/1293] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:34:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:34:24.354339 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec314b-a0c6-41c2-b20b-64fc2e1fcd68 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:24.354705 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec314b-a0c6-41c2-b20b-64fc2e1fcd68 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:24.354884 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec314b-a0c6-41c2-b20b-64fc2e1fcd68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:24.373188 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-2bec314b-a0c6-41c2-b20b-64fc2e1fcd68 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:34:24.374374 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-2bec314b-a0c6-41c2-b20b-64fc2e1fcd68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:34:24.402418 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:24.438927 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 652/1294] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:34:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:34:24.701613 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633268fc-c89c-4e83-8001-34e893e7cfc0 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:24.733567 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:24.812355 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633268fc-c89c-4e83-8001-34e893e7cfc0 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:24.844784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ceab03-4c29-47fc-8637-a2d4554ab1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:24.845376 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ceab03-4c29-47fc-8637-a2d4554ab1dc None admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:24.845649 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ceab03-4c29-47fc-8637-a2d4554ab1dc None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:24.845915 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ceab03-4c29-47fc-8637-a2d4554ab1dc None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:24.846542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ceab03-4c29-47fc-8637-a2d4554ab1dc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:24.847988 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ceab03-4c29-47fc-8637-a2d4554ab1dc None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:24.861765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 643/1295] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Aug 17 23:34:24 2022] POST /identity/v3/services => generated 246 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:34:26.756957 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7c2f2-db0a-46cd-bbde-500790717a72 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:26.757490 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7c2f2-db0a-46cd-bbde-500790717a72 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:26.757490 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7c2f2-db0a-46cd-bbde-500790717a72 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:26.758849 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 653/1296] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:34:26.763859 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba5654-0dde-4b92-a4b3-94cce7b3bad7 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:26.764275 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba5654-0dde-4b92-a4b3-94cce7b3bad7 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:26.764455 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba5654-0dde-4b92-a4b3-94cce7b3bad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:26.782732 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-2fba5654-0dde-4b92-a4b3-94cce7b3bad7 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:34:26.783912 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-2fba5654-0dde-4b92-a4b3-94cce7b3bad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:34:26.854116 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 644/1297] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:34:26 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:34:27.118953 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f7c2f2-db0a-46cd-bbde-500790717a72 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:27.216842 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f7c2f2-db0a-46cd-bbde-500790717a72 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:27.249409 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a7927f-24d8-40f3-b424-074f455280f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:27.250011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7927f-24d8-40f3-b424-074f455280f0 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:27.250285 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7927f-24d8-40f3-b424-074f455280f0 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:27.250550 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7927f-24d8-40f3-b424-074f455280f0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:27.251122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a7927f-24d8-40f3-b424-074f455280f0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:27.252512 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a7927f-24d8-40f3-b424-074f455280f0 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:27.260847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-a6a7927f-24d8-40f3-b424-074f455280f0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 17 23:34:27.260847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:34:27.260847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 23:34:27.260847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 23:34:27.260847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 23:34:27.260847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 23:34:27.260847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 23:34:27.260847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:34:27.260847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 17 23:34:27.260847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:34:27.260847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 23:34:27.260847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:34:27.260847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:34:27.260847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:34:27.260847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:34:27.260847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:34:27.260847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:34:27.262020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:34:27.262020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:34:27.262020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:34:27.262020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:34:27.262020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:34:27.262020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:34:27.262020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 23:34:27.262020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 23:34:27.262020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 23:34:27.262020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 23:34:27.262020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:34:27.262020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:34:27.262020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:34:27.262020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:34:27.262020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:34:27.262020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:34:27.263108 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:34:27.263108 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:34:27.263108 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:34:27.263108 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:34:27.263108 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:34:27.263108 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:34:27.263108 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:34:27.263108 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:34:27.263108 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:34:27.263108 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:34:27.263108 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 23:34:27.263108 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:34:27.263108 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 17 23:34:27.263108 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:34:27.263108 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 654/1298] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Aug 17 23:34:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:34:27.267819 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fba5654-0dde-4b92-a4b3-94cce7b3bad7 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:27.286041 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:27.363113 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fba5654-0dde-4b92-a4b3-94cce7b3bad7 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:27.395422 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcf3e23-a03b-4f26-b3e0-4a26aa3fa760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:27.395950 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf3e23-a03b-4f26-b3e0-4a26aa3fa760 None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:27.396252 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf3e23-a03b-4f26-b3e0-4a26aa3fa760 None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:27.396519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf3e23-a03b-4f26-b3e0-4a26aa3fa760 None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:27.397136 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcf3e23-a03b-4f26-b3e0-4a26aa3fa760 None admin] RBAC: Adding query filter params (name=placement) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:34:27.397433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcf3e23-a03b-4f26-b3e0-4a26aa3fa760 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:27.398785 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcf3e23-a03b-4f26-b3e0-4a26aa3fa760 None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:27.403919 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 645/1299] 10.222.0.26 () {56 vars in 1116 bytes} [Wed Aug 17 23:34:27 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:34:27.409638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a7927f-24d8-40f3-b424-074f455280f0 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:27.441976 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:27.507975 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a7927f-24d8-40f3-b424-074f455280f0 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:27.540364 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbeb424-48db-4502-936b-9f6a33f19ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:27.540963 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbeb424-48db-4502-936b-9f6a33f19ecc None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:27.541242 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbeb424-48db-4502-936b-9f6a33f19ecc None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:27.541515 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbeb424-48db-4502-936b-9f6a33f19ecc None admin] PATH_INFO: `/v3/endpoints` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:27.542211 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbeb424-48db-4502-936b-9f6a33f19ecc None admin] RBAC: Adding query filter params (interface=public, service_id=ed3055ae674845f68129e67fe3a9d783, region_id=RegionOne) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:34:27.542521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbeb424-48db-4502-936b-9f6a33f19ecc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:27.543884 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbeb424-48db-4502-936b-9f6a33f19ecc None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:27.548818 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 655/1300] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Aug 17 23:34:27 2022] GET /identity/v3/endpoints?service_id=ed3055ae674845f68129e67fe3a9d783&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:34:27.555007 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcf3e23-a03b-4f26-b3e0-4a26aa3fa760 None admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:27.632161 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:27.650455 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcf3e23-a03b-4f26-b3e0-4a26aa3fa760 None admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:27.682643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd35b8-9aea-4c83-8665-e64856649deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:27.683224 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd35b8-9aea-4c83-8665-e64856649deb None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:27.683497 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd35b8-9aea-4c83-8665-e64856649deb None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:27.683761 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd35b8-9aea-4c83-8665-e64856649deb None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:27.684386 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd35b8-9aea-4c83-8665-e64856649deb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:27.685734 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd35b8-9aea-4c83-8665-e64856649deb None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:27.692307 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 646/1301] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Aug 17 23:34:27 2022] GET /identity/v3/services => generated 2191 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:34:29.583264 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f041ca-8b26-4661-b8d0-ee838000ff4d None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:29.583802 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f041ca-8b26-4661-b8d0-ee838000ff4d None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:29.583802 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f041ca-8b26-4661-b8d0-ee838000ff4d None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:29.585192 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 656/1302] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:34:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:34:29.590402 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec37b9d9-50e3-4641-9db2-6592cb43d140 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:29.590806 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec37b9d9-50e3-4641-9db2-6592cb43d140 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:29.590953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec37b9d9-50e3-4641-9db2-6592cb43d140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:29.608386 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-ec37b9d9-50e3-4641-9db2-6592cb43d140 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:34:29.609553 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-ec37b9d9-50e3-4641-9db2-6592cb43d140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:34:29.678870 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 647/1303] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:34:29 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 23:34:29.943065 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f041ca-8b26-4661-b8d0-ee838000ff4d None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:30.034753 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:30.041347 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f041ca-8b26-4661-b8d0-ee838000ff4d None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:30.073396 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64490414-6b8b-45f6-8914-8162ca19f828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:30.074004 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64490414-6b8b-45f6-8914-8162ca19f828 None admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:30.074276 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64490414-6b8b-45f6-8914-8162ca19f828 None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:30.074541 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64490414-6b8b-45f6-8914-8162ca19f828 None admin] PATH_INFO: `/v3/services/placement` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:30.075111 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64490414-6b8b-45f6-8914-8162ca19f828 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:30.076497 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64490414-6b8b-45f6-8914-8162ca19f828 None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:30.085419 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application [None req-64490414-6b8b-45f6-8914-8162ca19f828 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 17 23:34:30.085419 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:34:30.085419 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 23:34:30.085419 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 23:34:30.085419 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 23:34:30.085419 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 23:34:30.085419 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 23:34:30.085419 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:34:30.085419 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 17 23:34:30.085419 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:34:30.085419 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 23:34:30.085419 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:34:30.085419 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 23:34:30.085419 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 23:34:30.085419 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 23:34:30.085419 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 23:34:30.085419 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 23:34:30.086638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 23:34:30.086638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 23:34:30.086638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 23:34:30.086638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 23:34:30.086638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 23:34:30.086638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 23:34:30.086638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 23:34:30.086638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 23:34:30.086638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 23:34:30.086638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 23:34:30.086638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 23:34:30.086638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 23:34:30.086638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 23:34:30.086638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 23:34:30.086638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 23:34:30.086638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 23:34:30.087833 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 23:34:30.087833 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application with Lock( Aug 17 23:34:30.087833 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 23:34:30.087833 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self._enter() Aug 17 23:34:30.087833 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 23:34:30.087833 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 23:34:30.087833 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 23:34:30.087833 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application return self.creator() Aug 17 23:34:30.087833 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 23:34:30.087833 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application created_value = creator( Aug 17 23:34:30.087833 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 23:34:30.087833 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 23:34:30.087833 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 17 23:34:30.087833 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: ERROR keystone.server.flask.application Aug 17 23:34:30.087833 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 657/1304] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Aug 17 23:34:29 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 23:34:30.092364 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec37b9d9-50e3-4641-9db2-6592cb43d140 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:30.165556 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:30.187443 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec37b9d9-50e3-4641-9db2-6592cb43d140 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:30.219865 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a528fbf3-afea-467f-9380-77d7bec2f6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:30.220483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528fbf3-afea-467f-9380-77d7bec2f6ca None admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:30.220763 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528fbf3-afea-467f-9380-77d7bec2f6ca None admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:30.221030 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528fbf3-afea-467f-9380-77d7bec2f6ca None admin] PATH_INFO: `/v3/services` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:30.221647 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a528fbf3-afea-467f-9380-77d7bec2f6ca None admin] RBAC: Adding query filter params (name=placement) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:34:30.221947 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a528fbf3-afea-467f-9380-77d7bec2f6ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:30.223852 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a528fbf3-afea-467f-9380-77d7bec2f6ca None admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:30.228868 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 648/1305] 10.222.0.26 () {56 vars in 1116 bytes} [Wed Aug 17 23:34:30 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:34:30.235445 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64490414-6b8b-45f6-8914-8162ca19f828 None admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:34:30.330605 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64490414-6b8b-45f6-8914-8162ca19f828 None admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:34:30.362695 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a1e49-bc9c-438f-9629-b63b390a352b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:34:30.363308 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a1e49-bc9c-438f-9629-b63b390a352b None admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:34:30.363584 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a1e49-bc9c-438f-9629-b63b390a352b None admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:34:30.363853 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a1e49-bc9c-438f-9629-b63b390a352b None admin] PATH_INFO: `/v3/endpoints` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:34:30.364529 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0a1e49-bc9c-438f-9629-b63b390a352b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:34:30.365879 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0a1e49-bc9c-438f-9629-b63b390a352b None admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:34:30.384595 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 658/1306] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:34:30 2022] POST /identity/v3/endpoints => generated 331 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:34:40.226005 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=30 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:34:40.380441 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:35:33.135186 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240bade8-6574-462f-ac48-a5aac2e51a6b None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:35:33.135641 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240bade8-6574-462f-ac48-a5aac2e51a6b None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:35:33.135930 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240bade8-6574-462f-ac48-a5aac2e51a6b None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:35:33.137368 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 649/1307] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:35:33.142607 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9aeae-8e0e-4e67-83d3-59a174f5743a None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:35:33.142998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9aeae-8e0e-4e67-83d3-59a174f5743a None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:35:33.143224 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9aeae-8e0e-4e67-83d3-59a174f5743a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:35:33.311322 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-72d9aeae-8e0e-4e67-83d3-59a174f5743a None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:35:33.313787 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-72d9aeae-8e0e-4e67-83d3-59a174f5743a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:35:33.484513 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 659/1308] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:35:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:35:33.778425 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f09aec-b02d-4d79-926b-4b4f4f3f14a7 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:35:33.778713 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f09aec-b02d-4d79-926b-4b4f4f3f14a7 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:35:33.778976 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f09aec-b02d-4d79-926b-4b4f4f3f14a7 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:35:33.780963 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 650/1309] 10.222.0.26 () {54 vars in 965 bytes} [Wed Aug 17 23:35:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:35:33.788016 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a1b85-f720-485c-a637-604edc02eac2 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:35:33.788336 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a1b85-f720-485c-a637-604edc02eac2 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:35:33.788584 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a1b85-f720-485c-a637-604edc02eac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:35:33.877788 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-f05a1b85-f720-485c-a637-604edc02eac2 None None] MFA Rules not processed for user `16e2758f1cd144a98b98d9be0fa46ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:35:33.879241 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-f05a1b85-f720-485c-a637-604edc02eac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:35:33.903114 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:35:33.967181 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 660/1310] 10.222.0.26 () {58 vars in 1062 bytes} [Wed Aug 17 23:35:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:35:33.976900 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f09aec-b02d-4d79-926b-4b4f4f3f14a7 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:35:33.979016 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-b6f09aec-b02d-4d79-926b-4b4f4f3f14a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:35:34.089385 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f09aec-b02d-4d79-926b-4b4f4f3f14a7 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:35:34.164717 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8dc058-cf11-4bd5-b07e-33677dad9613 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:35:34.165354 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8dc058-cf11-4bd5-b07e-33677dad9613 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:35:34.165588 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8dc058-cf11-4bd5-b07e-33677dad9613 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:35:34.165837 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8dc058-cf11-4bd5-b07e-33677dad9613 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:35:34.167439 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-fa8dc058-cf11-4bd5-b07e-33677dad9613 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:35:34.224879 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:35:34.281806 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8dc058-cf11-4bd5-b07e-33677dad9613 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:35:34.283770 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8dc058-cf11-4bd5-b07e-33677dad9613 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:35:34.303300 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 651/1311] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:35:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:35:36.475318 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9007939-4703-4d1c-ac89-d252a423b303 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:35:36.475758 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9007939-4703-4d1c-ac89-d252a423b303 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:35:36.475904 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9007939-4703-4d1c-ac89-d252a423b303 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:35:36.477296 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 661/1312] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:35:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:35:36.482605 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92f033-42ed-48f8-9dca-44f399e9d334 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:35:36.483019 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92f033-42ed-48f8-9dca-44f399e9d334 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:35:36.483148 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92f033-42ed-48f8-9dca-44f399e9d334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:35:36.556747 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-db92f033-42ed-48f8-9dca-44f399e9d334 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:35:36.557900 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-db92f033-42ed-48f8-9dca-44f399e9d334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:35:36.658532 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 652/1313] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:35:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:35:36.935593 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9007939-4703-4d1c-ac89-d252a423b303 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:35:36.937180 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-c9007939-4703-4d1c-ac89-d252a423b303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:35:37.038196 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9007939-4703-4d1c-ac89-d252a423b303 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:35:37.107035 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:35:37.109728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a2f71-81ae-45e2-ae4a-74e778e30aca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:35:37.110625 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a2f71-81ae-45e2-ae4a-74e778e30aca service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:35:37.111623 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a2f71-81ae-45e2-ae4a-74e778e30aca service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:35:37.112097 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a2f71-81ae-45e2-ae4a-74e778e30aca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:35:37.113700 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-7d7a2f71-81ae-45e2-ae4a-74e778e30aca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:35:37.219253 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7a2f71-81ae-45e2-ae4a-74e778e30aca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:35:37.220742 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7a2f71-81ae-45e2-ae4a-74e778e30aca service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:35:37.238594 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 662/1314] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:35:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:35:39.424268 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f2e49-6f84-4233-a6ab-eec549e89523 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:35:39.424755 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f2e49-6f84-4233-a6ab-eec549e89523 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:35:39.424852 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f2e49-6f84-4233-a6ab-eec549e89523 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:35:39.426131 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 653/1315] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:35:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:35:39.431259 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee698a-89a1-4895-bf56-429aba6a749e None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:35:39.431678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee698a-89a1-4895-bf56-429aba6a749e None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:35:39.431803 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee698a-89a1-4895-bf56-429aba6a749e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:35:39.501483 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-e9ee698a-89a1-4895-bf56-429aba6a749e None None] MFA Rules not processed for user `45f6f1fc05fd4bf590127b863244444f`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:35:39.502709 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-e9ee698a-89a1-4895-bf56-429aba6a749e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:35:39.553033 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:35:39.597111 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 663/1316] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:35:39 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:35:39.878239 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f2e49-6f84-4233-a6ab-eec549e89523 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:35:39.879877 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-078f2e49-6f84-4233-a6ab-eec549e89523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:35:39.889416 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:35:39.982696 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f2e49-6f84-4233-a6ab-eec549e89523 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:35:40.054894 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1780e64d-5314-446f-9498-7ab273eca18b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:35:40.055485 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780e64d-5314-446f-9498-7ab273eca18b service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:35:40.055758 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780e64d-5314-446f-9498-7ab273eca18b service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:35:40.056052 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780e64d-5314-446f-9498-7ab273eca18b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:35:40.057621 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-1780e64d-5314-446f-9498-7ab273eca18b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:35:40.117130 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:35:40.162979 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1780e64d-5314-446f-9498-7ab273eca18b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:35:40.164362 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1780e64d-5314-446f-9498-7ab273eca18b service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:35:40.182678 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 654/1317] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:35:39 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:35:42.614446 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc74bdb-d170-4cdd-b57a-cf41d076d022 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:35:42.614934 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc74bdb-d170-4cdd-b57a-cf41d076d022 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:35:42.615023 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc74bdb-d170-4cdd-b57a-cf41d076d022 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:35:42.616367 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 664/1318] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:35:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:35:42.621434 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95f4df-b979-40f2-a6c5-7d3844471646 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:35:42.621826 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95f4df-b979-40f2-a6c5-7d3844471646 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:35:42.621973 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95f4df-b979-40f2-a6c5-7d3844471646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:35:42.685998 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-9e95f4df-b979-40f2-a6c5-7d3844471646 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:35:42.687218 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-9e95f4df-b979-40f2-a6c5-7d3844471646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:35:42.783476 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 655/1319] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:35:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:35:43.068090 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc74bdb-d170-4cdd-b57a-cf41d076d022 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:35:43.069636 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-0cc74bdb-d170-4cdd-b57a-cf41d076d022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:35:43.170407 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc74bdb-d170-4cdd-b57a-cf41d076d022 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:35:43.242093 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05a288f-3ced-43f5-8d36-3ccb5125ebd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:35:43.242694 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a288f-3ced-43f5-8d36-3ccb5125ebd6 service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:35:43.242973 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a288f-3ced-43f5-8d36-3ccb5125ebd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:35:43.243244 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a288f-3ced-43f5-8d36-3ccb5125ebd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:35:43.244860 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-d05a288f-3ced-43f5-8d36-3ccb5125ebd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:35:43.252092 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:35:43.352518 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05a288f-3ced-43f5-8d36-3ccb5125ebd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:35:43.353884 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05a288f-3ced-43f5-8d36-3ccb5125ebd6 service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:35:43.371642 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 665/1320] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:35:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:35:45.892294 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c3d12c-26ac-47a5-820f-67f8b89b1512 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:35:45.892855 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c3d12c-26ac-47a5-820f-67f8b89b1512 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:35:45.892855 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c3d12c-26ac-47a5-820f-67f8b89b1512 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:35:45.894182 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 656/1321] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:35:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:35:45.899414 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac88ff0-595c-49be-8ddf-838fc145f975 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:35:45.899832 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac88ff0-595c-49be-8ddf-838fc145f975 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:35:45.900018 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac88ff0-595c-49be-8ddf-838fc145f975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:35:45.965044 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-3ac88ff0-595c-49be-8ddf-838fc145f975 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:35:45.966238 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-3ac88ff0-595c-49be-8ddf-838fc145f975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:35:46.023998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:35:46.063215 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 666/1322] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:35:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:35:46.359137 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702f969-eb88-4ab8-bb12-f53b5c27fe9d None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:35:46.359424 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702f969-eb88-4ab8-bb12-f53b5c27fe9d None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:35:46.359699 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702f969-eb88-4ab8-bb12-f53b5c27fe9d None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:35:46.361119 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 657/1323] 10.222.0.26 () {54 vars in 965 bytes} [Wed Aug 17 23:35:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:35:46.367474 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d3b94-ef42-42a4-9a01-4003fe7e3bb1 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:35:46.367797 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d3b94-ef42-42a4-9a01-4003fe7e3bb1 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:35:46.368113 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d3b94-ef42-42a4-9a01-4003fe7e3bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:35:46.445434 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-a80d3b94-ef42-42a4-9a01-4003fe7e3bb1 None None] MFA Rules not processed for user `16e2758f1cd144a98b98d9be0fa46ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:35:46.446644 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-a80d3b94-ef42-42a4-9a01-4003fe7e3bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:35:46.524038 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 667/1324] 10.222.0.26 () {58 vars in 1062 bytes} [Wed Aug 17 23:35:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:35:46.533836 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f702f969-eb88-4ab8-bb12-f53b5c27fe9d None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:35:46.535231 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-f702f969-eb88-4ab8-bb12-f53b5c27fe9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:35:46.554284 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:35:46.639754 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f702f969-eb88-4ab8-bb12-f53b5c27fe9d None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:35:46.716877 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9e56af-389d-41ba-afb5-dfeaaca2cc18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:35:46.717478 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e56af-389d-41ba-afb5-dfeaaca2cc18 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:35:46.717754 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e56af-389d-41ba-afb5-dfeaaca2cc18 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:35:46.718022 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e56af-389d-41ba-afb5-dfeaaca2cc18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:35:46.719607 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-8c9e56af-389d-41ba-afb5-dfeaaca2cc18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:35:46.790713 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:35:46.828399 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9e56af-389d-41ba-afb5-dfeaaca2cc18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:35:46.830439 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9e56af-389d-41ba-afb5-dfeaaca2cc18 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:35:46.849697 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 658/1325] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:35:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:35:49.019860 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818777da-90aa-4fd1-885f-e072c760988a None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:35:49.020451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818777da-90aa-4fd1-885f-e072c760988a None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:35:49.020451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818777da-90aa-4fd1-885f-e072c760988a None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:35:49.021779 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 668/1326] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:35:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:35:49.026843 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5231290c-b4a3-41d8-a2d7-431efe40699d None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:35:49.027271 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5231290c-b4a3-41d8-a2d7-431efe40699d None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:35:49.027414 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5231290c-b4a3-41d8-a2d7-431efe40699d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:35:49.092626 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-5231290c-b4a3-41d8-a2d7-431efe40699d None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:35:49.093822 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-5231290c-b4a3-41d8-a2d7-431efe40699d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:35:49.201376 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 659/1327] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:35:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:35:49.483022 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818777da-90aa-4fd1-885f-e072c760988a None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:35:49.484580 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-818777da-90aa-4fd1-885f-e072c760988a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:35:49.546301 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:35:49.590644 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818777da-90aa-4fd1-885f-e072c760988a None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:35:49.663720 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309a509b-6270-45bc-93f0-55175e2f80a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:35:49.664354 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a509b-6270-45bc-93f0-55175e2f80a2 service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:35:49.664632 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a509b-6270-45bc-93f0-55175e2f80a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:35:49.664908 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a509b-6270-45bc-93f0-55175e2f80a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:35:49.666498 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-309a509b-6270-45bc-93f0-55175e2f80a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:35:49.773437 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:35:49.775788 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309a509b-6270-45bc-93f0-55175e2f80a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:35:49.777576 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309a509b-6270-45bc-93f0-55175e2f80a2 service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:35:49.797056 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 669/1328] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:35:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:35:51.759555 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dee8f3-7191-471f-979e-ee7ae9836582 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:35:51.760148 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dee8f3-7191-471f-979e-ee7ae9836582 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:35:51.760148 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dee8f3-7191-471f-979e-ee7ae9836582 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:35:51.761472 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 660/1329] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:35:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:35:51.767497 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db2c121-3ad2-4b5e-b950-4f7a18da0d55 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:35:51.768035 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db2c121-3ad2-4b5e-b950-4f7a18da0d55 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:35:51.768260 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db2c121-3ad2-4b5e-b950-4f7a18da0d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:35:51.839449 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-4db2c121-3ad2-4b5e-b950-4f7a18da0d55 None None] MFA Rules not processed for user `45f6f1fc05fd4bf590127b863244444f`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:35:51.840714 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-4db2c121-3ad2-4b5e-b950-4f7a18da0d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:35:51.934735 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 670/1330] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:35:51 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:35:52.223182 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dee8f3-7191-471f-979e-ee7ae9836582 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:35:52.224832 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-29dee8f3-7191-471f-979e-ee7ae9836582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:35:52.255283 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:35:52.330950 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dee8f3-7191-471f-979e-ee7ae9836582 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:35:52.408248 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf229b3-bc9d-459a-97af-e7eb04159df1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:35:52.408860 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf229b3-bc9d-459a-97af-e7eb04159df1 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:35:52.409136 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf229b3-bc9d-459a-97af-e7eb04159df1 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:35:52.409404 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf229b3-bc9d-459a-97af-e7eb04159df1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:35:52.411028 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-ebf229b3-bc9d-459a-97af-e7eb04159df1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:35:52.494489 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:35:52.527572 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf229b3-bc9d-459a-97af-e7eb04159df1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:35:52.529334 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf229b3-bc9d-459a-97af-e7eb04159df1 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:35:52.551740 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 661/1331] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:35:52 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:35:54.865655 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ff169-488c-48ac-b1d5-6062d95c1cad None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:35:54.866192 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ff169-488c-48ac-b1d5-6062d95c1cad None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:35:54.866192 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ff169-488c-48ac-b1d5-6062d95c1cad None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:35:54.867549 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 671/1332] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:35:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:35:54.872539 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79fb3dd-8cdc-4404-be1b-dce6bdbc801d None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:35:54.872908 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79fb3dd-8cdc-4404-be1b-dce6bdbc801d None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:35:54.873078 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79fb3dd-8cdc-4404-be1b-dce6bdbc801d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:35:54.937367 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-b79fb3dd-8cdc-4404-be1b-dce6bdbc801d None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:35:54.939196 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-b79fb3dd-8cdc-4404-be1b-dce6bdbc801d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:35:55.034012 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 662/1333] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:35:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:35:55.311119 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ff169-488c-48ac-b1d5-6062d95c1cad None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:35:55.312674 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-e83ff169-488c-48ac-b1d5-6062d95c1cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:35:55.381541 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:35:55.416381 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ff169-488c-48ac-b1d5-6062d95c1cad None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:35:55.488208 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2c171f-8d30-42b9-81cf-6771b7e35c15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:35:55.488814 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c171f-8d30-42b9-81cf-6771b7e35c15 service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:35:55.489093 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c171f-8d30-42b9-81cf-6771b7e35c15 service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:35:55.489366 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c171f-8d30-42b9-81cf-6771b7e35c15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:35:55.490971 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-ff2c171f-8d30-42b9-81cf-6771b7e35c15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:35:55.598329 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2c171f-8d30-42b9-81cf-6771b7e35c15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:35:55.599691 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2c171f-8d30-42b9-81cf-6771b7e35c15 service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:35:55.609567 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:35:55.617967 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 672/1334] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:35:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:35:57.537772 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f1ff8-460e-4235-885e-280d968a72c0 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:35:57.538315 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f1ff8-460e-4235-885e-280d968a72c0 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:35:57.538315 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f1ff8-460e-4235-885e-280d968a72c0 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:35:57.539650 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 663/1335] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:35:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:35:57.544658 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3ef7b-1126-46a6-b23a-96c99390369d None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:35:57.545069 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3ef7b-1126-46a6-b23a-96c99390369d None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:35:57.545223 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3ef7b-1126-46a6-b23a-96c99390369d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:35:57.610856 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-d6d3ef7b-1126-46a6-b23a-96c99390369d None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:35:57.612103 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-d6d3ef7b-1126-46a6-b23a-96c99390369d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:35:57.706139 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 673/1336] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:35:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:35:57.990870 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f1ff8-460e-4235-885e-280d968a72c0 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:35:57.992558 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-9e9f1ff8-460e-4235-885e-280d968a72c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:35:58.033065 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:35:58.096103 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f1ff8-460e-4235-885e-280d968a72c0 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:35:58.167235 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ab0f30-1ad5-4c8d-ba89-0a2af0d8cb49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:35:58.167837 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab0f30-1ad5-4c8d-ba89-0a2af0d8cb49 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:35:58.168154 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab0f30-1ad5-4c8d-ba89-0a2af0d8cb49 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:35:58.168426 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab0f30-1ad5-4c8d-ba89-0a2af0d8cb49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:35:58.170041 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-00ab0f30-1ad5-4c8d-ba89-0a2af0d8cb49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:35:58.250950 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:35:58.276232 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ab0f30-1ad5-4c8d-ba89-0a2af0d8cb49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:35:58.277676 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ab0f30-1ad5-4c8d-ba89-0a2af0d8cb49 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:35:58.296421 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 664/1337] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:35:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:36:00.521505 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6dd8d-06e7-48bc-8f9e-12bc0b795853 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:00.522008 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6dd8d-06e7-48bc-8f9e-12bc0b795853 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:00.522101 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6dd8d-06e7-48bc-8f9e-12bc0b795853 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:00.523384 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 674/1338] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:36:00.528646 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68edc6-c132-4a2d-acbe-87c02083e36b None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:00.529069 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68edc6-c132-4a2d-acbe-87c02083e36b None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:00.529175 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68edc6-c132-4a2d-acbe-87c02083e36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:00.592812 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-5c68edc6-c132-4a2d-acbe-87c02083e36b None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:36:00.593985 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-5c68edc6-c132-4a2d-acbe-87c02083e36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:00.686801 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 665/1339] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:36:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:36:00.970175 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa6dd8d-06e7-48bc-8f9e-12bc0b795853 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:36:00.971695 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-aaa6dd8d-06e7-48bc-8f9e-12bc0b795853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:01.044981 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:01.072207 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa6dd8d-06e7-48bc-8f9e-12bc0b795853 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:36:01.142885 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa910e0-c0ca-40c6-a441-528d4013e357 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:36:01.143488 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa910e0-c0ca-40c6-a441-528d4013e357 service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:01.143771 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa910e0-c0ca-40c6-a441-528d4013e357 service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:01.144072 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa910e0-c0ca-40c6-a441-528d4013e357 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:01.145678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-0fa910e0-c0ca-40c6-a441-528d4013e357 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:01.251372 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa910e0-c0ca-40c6-a441-528d4013e357 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:36:01.252842 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa910e0-c0ca-40c6-a441-528d4013e357 service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:36:01.270584 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 675/1340] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:36:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:36:05.083596 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a118e-c3f8-44cf-ac85-bdd04cedbe6d None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:05.084034 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a118e-c3f8-44cf-ac85-bdd04cedbe6d None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:05.084199 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a118e-c3f8-44cf-ac85-bdd04cedbe6d None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:05.085555 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 666/1341] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:36:05.090899 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31dc971-20d7-4dc3-8564-c6290347eac7 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:05.091326 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31dc971-20d7-4dc3-8564-c6290347eac7 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:05.091580 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31dc971-20d7-4dc3-8564-c6290347eac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:05.097656 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:05.161352 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-b31dc971-20d7-4dc3-8564-c6290347eac7 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:36:05.162570 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-b31dc971-20d7-4dc3-8564-c6290347eac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:05.260785 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 676/1342] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:36:05.541574 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27a118e-c3f8-44cf-ac85-bdd04cedbe6d None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:36:05.543137 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-b27a118e-c3f8-44cf-ac85-bdd04cedbe6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:05.591249 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:05.648800 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27a118e-c3f8-44cf-ac85-bdd04cedbe6d None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:36:05.721792 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1c4de-866c-43dc-9e9e-b9f0032a4acf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:36:05.722399 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1c4de-866c-43dc-9e9e-b9f0032a4acf service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:05.722673 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1c4de-866c-43dc-9e9e-b9f0032a4acf service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:05.722941 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1c4de-866c-43dc-9e9e-b9f0032a4acf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:05.724554 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-6ca1c4de-866c-43dc-9e9e-b9f0032a4acf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:05.816548 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:05.833947 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca1c4de-866c-43dc-9e9e-b9f0032a4acf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:36:05.835377 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca1c4de-866c-43dc-9e9e-b9f0032a4acf service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:36:05.854260 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 667/1343] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:36:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:36:08.424764 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d65c8-a22c-4b52-97f0-a962bc6bf38e None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:08.425341 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d65c8-a22c-4b52-97f0-a962bc6bf38e None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:08.425341 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d65c8-a22c-4b52-97f0-a962bc6bf38e None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:08.426678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 677/1344] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:36:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:36:08.432236 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f832e876-18a0-46f1-9258-2b83240de05d None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:08.432636 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f832e876-18a0-46f1-9258-2b83240de05d None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:08.432782 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f832e876-18a0-46f1-9258-2b83240de05d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:08.504674 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-f832e876-18a0-46f1-9258-2b83240de05d None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:36:08.505915 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-f832e876-18a0-46f1-9258-2b83240de05d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:08.603854 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 668/1345] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:36:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:36:08.890314 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0d65c8-a22c-4b52-97f0-a962bc6bf38e None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:36:08.892483 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-fa0d65c8-a22c-4b52-97f0-a962bc6bf38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:08.978572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:08.998886 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0d65c8-a22c-4b52-97f0-a962bc6bf38e None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:36:09.073034 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e612139c-2423-4316-a7c4-3d8f5245f90d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:36:09.073634 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612139c-2423-4316-a7c4-3d8f5245f90d service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:09.073908 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612139c-2423-4316-a7c4-3d8f5245f90d service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:09.074175 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612139c-2423-4316-a7c4-3d8f5245f90d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:09.075756 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-e612139c-2423-4316-a7c4-3d8f5245f90d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:09.188764 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e612139c-2423-4316-a7c4-3d8f5245f90d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:36:09.190119 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e612139c-2423-4316-a7c4-3d8f5245f90d service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:36:09.207625 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 678/1346] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:36:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:36:13.089073 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb7f45-2a7b-446d-b7d7-94fe8afb5a9d None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:13.089584 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb7f45-2a7b-446d-b7d7-94fe8afb5a9d None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:13.089679 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb7f45-2a7b-446d-b7d7-94fe8afb5a9d None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:13.090972 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 669/1347] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:36:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:36:13.097033 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21632402-ee57-48f7-a87a-398d203f5b52 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:13.097505 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21632402-ee57-48f7-a87a-398d203f5b52 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:13.097599 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21632402-ee57-48f7-a87a-398d203f5b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:13.114729 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:13.187034 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-21632402-ee57-48f7-a87a-398d203f5b52 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:36:13.188425 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-21632402-ee57-48f7-a87a-398d203f5b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:13.295842 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 679/1348] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:36:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:36:13.573866 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb7f45-2a7b-446d-b7d7-94fe8afb5a9d None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:36:13.575393 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-39eb7f45-2a7b-446d-b7d7-94fe8afb5a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:13.634660 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:13.681191 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb7f45-2a7b-446d-b7d7-94fe8afb5a9d None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:36:13.759198 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7771ddbe-cb95-452d-b26f-1d4e0d87bf5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:36:13.759908 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7771ddbe-cb95-452d-b26f-1d4e0d87bf5a service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:13.760201 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7771ddbe-cb95-452d-b26f-1d4e0d87bf5a service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:13.760470 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7771ddbe-cb95-452d-b26f-1d4e0d87bf5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:13.762071 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-7771ddbe-cb95-452d-b26f-1d4e0d87bf5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:13.866762 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:13.870854 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7771ddbe-cb95-452d-b26f-1d4e0d87bf5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:36:13.872346 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7771ddbe-cb95-452d-b26f-1d4e0d87bf5a service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:36:13.890297 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 670/1349] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:36:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:36:18.743603 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0fa69-7650-4817-93fb-b1e9c885c773 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:18.744136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0fa69-7650-4817-93fb-b1e9c885c773 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:18.744248 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0fa69-7650-4817-93fb-b1e9c885c773 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:18.747226 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 680/1350] 10.222.0.26 () {54 vars in 921 bytes} [Wed Aug 17 23:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:36:18.753556 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745fb75-580b-4e07-8495-7a8d93de6bf1 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:18.754000 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745fb75-580b-4e07-8495-7a8d93de6bf1 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:18.754179 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745fb75-580b-4e07-8495-7a8d93de6bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:18.860791 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-5745fb75-580b-4e07-8495-7a8d93de6bf1 None None] MFA Rules not processed for user `1fc57d911a56423e96c736bc96270ba8`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:36:18.862237 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-5745fb75-580b-4e07-8495-7a8d93de6bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:19.033166 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 671/1351] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Aug 17 23:36:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:36:23.193551 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea5d7a-517f-4425-a6c8-ac57dc25bf78 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:23.194136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea5d7a-517f-4425-a6c8-ac57dc25bf78 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:23.194136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea5d7a-517f-4425-a6c8-ac57dc25bf78 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:23.195605 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 681/1352] 10.222.0.26 () {54 vars in 921 bytes} [Wed Aug 17 23:36:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:36:23.200562 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388e74e-f87e-4128-a0e0-8d675d32285a None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:23.201021 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388e74e-f87e-4128-a0e0-8d675d32285a None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:23.201145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388e74e-f87e-4128-a0e0-8d675d32285a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:23.255358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:23.280099 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-8388e74e-f87e-4128-a0e0-8d675d32285a None None] MFA Rules not processed for user `1fc57d911a56423e96c736bc96270ba8`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:36:23.281847 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-8388e74e-f87e-4128-a0e0-8d675d32285a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:23.293908 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=59 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:23.393858 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 672/1353] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Aug 17 23:36:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:36:24.782966 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185246a9-2447-45c5-a2bb-d0522a04981c None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:24.783595 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185246a9-2447-45c5-a2bb-d0522a04981c None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:24.783595 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185246a9-2447-45c5-a2bb-d0522a04981c None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:24.787674 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 682/1354] 10.222.0.26 () {54 vars in 921 bytes} [Wed Aug 17 23:36:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:36:24.792691 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6d614-3a65-4ca4-99db-e4c57a721832 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:24.793133 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6d614-3a65-4ca4-99db-e4c57a721832 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:24.793240 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6d614-3a65-4ca4-99db-e4c57a721832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:24.870774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-dae6d614-3a65-4ca4-99db-e4c57a721832 None None] MFA Rules not processed for user `1fc57d911a56423e96c736bc96270ba8`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:36:24.872007 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-dae6d614-3a65-4ca4-99db-e4c57a721832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:24.915823 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:24.981331 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 673/1355] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Aug 17 23:36:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:36:28.219633 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd723791-2b62-40e6-9a76-2193827a16b2 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:28.220198 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd723791-2b62-40e6-9a76-2193827a16b2 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:28.220198 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd723791-2b62-40e6-9a76-2193827a16b2 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:28.221602 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 683/1356] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:36:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:36:28.226767 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d0ae8-4767-4977-a29c-bc6f4af1bf69 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:28.227164 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d0ae8-4767-4977-a29c-bc6f4af1bf69 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:28.227343 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d0ae8-4767-4977-a29c-bc6f4af1bf69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:28.309423 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-ed4d0ae8-4767-4977-a29c-bc6f4af1bf69 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:36:28.310599 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-ed4d0ae8-4767-4977-a29c-bc6f4af1bf69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:28.440013 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 674/1357] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:36:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:36:28.746585 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd5010-916f-458b-bf37-342561d852d4 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:28.747152 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd5010-916f-458b-bf37-342561d852d4 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:28.747672 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd5010-916f-458b-bf37-342561d852d4 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:28.750942 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 684/1358] 10.222.0.26 () {54 vars in 962 bytes} [Wed Aug 17 23:36:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:36:28.759311 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2249d-34e5-4734-b485-cc076f1d76df None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:28.759606 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2249d-34e5-4734-b485-cc076f1d76df None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:28.759889 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2249d-34e5-4734-b485-cc076f1d76df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:28.782278 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:28.846334 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-fce2249d-34e5-4734-b485-cc076f1d76df None None] MFA Rules not processed for user `ec1dcbe43b0749c3bcc4af9100d771d1`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:36:28.847448 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-fce2249d-34e5-4734-b485-cc076f1d76df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:28.973727 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 675/1359] 10.222.0.26 () {58 vars in 1059 bytes} [Wed Aug 17 23:36:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:36:28.981372 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd5010-916f-458b-bf37-342561d852d4 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:36:28.983015 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-30fd5010-916f-458b-bf37-342561d852d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:29.148161 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd5010-916f-458b-bf37-342561d852d4 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:36:29.229884 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:29.268727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec104ee-223b-4df9-95a1-3f1c89edd4e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:36:29.269520 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec104ee-223b-4df9-95a1-3f1c89edd4e1 service nova] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:29.269898 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec104ee-223b-4df9-95a1-3f1c89edd4e1 service nova] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:29.270333 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec104ee-223b-4df9-95a1-3f1c89edd4e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:29.272020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-fec104ee-223b-4df9-95a1-3f1c89edd4e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:29.363329 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec104ee-223b-4df9-95a1-3f1c89edd4e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:36:29.364998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec104ee-223b-4df9-95a1-3f1c89edd4e1 service nova] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:36:29.383378 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 685/1360] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:36:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:36:31.476681 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e44f6-bb39-4bcb-9474-37141a09167d None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:31.477202 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e44f6-bb39-4bcb-9474-37141a09167d None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:31.477202 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e44f6-bb39-4bcb-9474-37141a09167d None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:31.478718 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 676/1361] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:36:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:36:31.485050 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfce767-2262-416a-a6b4-ba0b37020728 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:31.485616 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfce767-2262-416a-a6b4-ba0b37020728 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:31.485616 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfce767-2262-416a-a6b4-ba0b37020728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:31.554389 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-ddfce767-2262-416a-a6b4-ba0b37020728 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:36:31.555635 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-ddfce767-2262-416a-a6b4-ba0b37020728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:31.594645 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:31.659268 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 686/1362] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:36:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:36:31.950619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051e44f6-bb39-4bcb-9474-37141a09167d None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:36:31.952046 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-051e44f6-bb39-4bcb-9474-37141a09167d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:32.026365 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:32.117793 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051e44f6-bb39-4bcb-9474-37141a09167d None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:36:32.233975 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49251d-f56b-4ed4-bcce-fa7e5f8436ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:36:32.235213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49251d-f56b-4ed4-bcce-fa7e5f8436ee service nova] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:32.235493 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49251d-f56b-4ed4-bcce-fa7e5f8436ee service nova] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:32.235761 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49251d-f56b-4ed4-bcce-fa7e5f8436ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:32.237409 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-ac49251d-f56b-4ed4-bcce-fa7e5f8436ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:32.270420 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:32.351770 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49251d-f56b-4ed4-bcce-fa7e5f8436ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:36:32.353440 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49251d-f56b-4ed4-bcce-fa7e5f8436ee service nova] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:36:32.373553 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 677/1363] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:36:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:36:34.596002 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae495f-4cfc-48d0-9720-baed243139a8 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:34.596864 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae495f-4cfc-48d0-9720-baed243139a8 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:34.597140 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae495f-4cfc-48d0-9720-baed243139a8 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:34.598759 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 687/1364] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:36:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:36:34.607868 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c208e-78cf-402e-a6d3-d228a2690b68 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:34.609721 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c208e-78cf-402e-a6d3-d228a2690b68 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:34.610017 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c208e-78cf-402e-a6d3-d228a2690b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:34.685591 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-4c2c208e-78cf-402e-a6d3-d228a2690b68 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:36:34.686818 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-4c2c208e-78cf-402e-a6d3-d228a2690b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:34.787126 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:34.789971 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 678/1365] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:36:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:36:35.081285 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ecae09-196a-4285-858c-d9a965f3bfd1 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:35.081763 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ecae09-196a-4285-858c-d9a965f3bfd1 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:35.082136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ecae09-196a-4285-858c-d9a965f3bfd1 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:35.083867 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 688/1366] 10.222.0.26 () {54 vars in 962 bytes} [Wed Aug 17 23:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:36:35.090023 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d89e42-b594-4326-8572-3ab4c5046c06 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:35.090378 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d89e42-b594-4326-8572-3ab4c5046c06 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:35.090585 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d89e42-b594-4326-8572-3ab4c5046c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:35.170099 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-91d89e42-b594-4326-8572-3ab4c5046c06 None None] MFA Rules not processed for user `ec1dcbe43b0749c3bcc4af9100d771d1`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:36:35.171366 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-91d89e42-b594-4326-8572-3ab4c5046c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:35.296614 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 679/1367] 10.222.0.26 () {58 vars in 1059 bytes} [Wed Aug 17 23:36:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:36:35.305184 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ecae09-196a-4285-858c-d9a965f3bfd1 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:36:35.306800 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-31ecae09-196a-4285-858c-d9a965f3bfd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:35.462910 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ecae09-196a-4285-858c-d9a965f3bfd1 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:36:35.484311 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:35.586900 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a516ef-ee98-44c1-9ba6-231b8d93efc8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:36:35.587662 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a516ef-ee98-44c1-9ba6-231b8d93efc8 service nova] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:35.588074 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a516ef-ee98-44c1-9ba6-231b8d93efc8 service nova] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:35.588444 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a516ef-ee98-44c1-9ba6-231b8d93efc8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:35.590182 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-90a516ef-ee98-44c1-9ba6-231b8d93efc8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:35.681762 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a516ef-ee98-44c1-9ba6-231b8d93efc8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:36:35.684107 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a516ef-ee98-44c1-9ba6-231b8d93efc8 service nova] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:36:35.704412 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 689/1368] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:36:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:36:36.394642 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cea3e3-e329-45d1-8176-2c13a1abc31b None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:36.395173 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cea3e3-e329-45d1-8176-2c13a1abc31b None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:36.395173 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cea3e3-e329-45d1-8176-2c13a1abc31b None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:36.400006 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 680/1369] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:36:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:36:36.402136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7c949-038a-4c3a-bfcf-80588544e1e8 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:36.402528 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7c949-038a-4c3a-bfcf-80588544e1e8 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:36.402946 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7c949-038a-4c3a-bfcf-80588544e1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:36.437945 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:36.482666 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-eae7c949-038a-4c3a-bfcf-80588544e1e8 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:36:36.484648 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-eae7c949-038a-4c3a-bfcf-80588544e1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:36.590795 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 690/1370] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:36:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:36:37.186742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60beedb0-78f3-43c0-89b5-153836c15a22 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:37.186742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60beedb0-78f3-43c0-89b5-153836c15a22 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:37.186742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60beedb0-78f3-43c0-89b5-153836c15a22 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:37.187935 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 681/1371] 10.222.0.26 () {54 vars in 964 bytes} [Wed Aug 17 23:36:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:36:37.192406 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93394ada-f08d-40fe-8555-71b47a4c104c None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:37.192855 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93394ada-f08d-40fe-8555-71b47a4c104c None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:37.192950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93394ada-f08d-40fe-8555-71b47a4c104c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:37.283603 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-93394ada-f08d-40fe-8555-71b47a4c104c None None] MFA Rules not processed for user `62906d50b6e247c59b69809d3959d4de`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:36:37.285242 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-93394ada-f08d-40fe-8555-71b47a4c104c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:37.332963 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:37.368183 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 691/1372] 10.222.0.26 () {58 vars in 1061 bytes} [Wed Aug 17 23:36:37 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:36:37.372045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60beedb0-78f3-43c0-89b5-153836c15a22 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:36:37.373815 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-60beedb0-78f3-43c0-89b5-153836c15a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:37.434107 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:37.505942 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60beedb0-78f3-43c0-89b5-153836c15a22 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:36:37.642890 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614717c-15c4-4283-abe4-acb8ddf8a248 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:36:37.644503 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614717c-15c4-4283-abe4-acb8ddf8a248 service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:37.645283 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614717c-15c4-4283-abe4-acb8ddf8a248 service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:37.646074 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614717c-15c4-4283-abe4-acb8ddf8a248 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:37.649431 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-0614717c-15c4-4283-abe4-acb8ddf8a248 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:37.706132 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7421e-d4e0-4826-8b03-74de32b7ffe6 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:37.706428 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7421e-d4e0-4826-8b03-74de32b7ffe6 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:37.706709 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7421e-d4e0-4826-8b03-74de32b7ffe6 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:37.708386 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 692/1373] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:36:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 23:36:37.713231 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc892c-1d16-4271-9b7b-6ecfae514224 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:37.713513 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc892c-1d16-4271-9b7b-6ecfae514224 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:37.713772 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc892c-1d16-4271-9b7b-6ecfae514224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:37.799179 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-0fdc892c-1d16-4271-9b7b-6ecfae514224 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:36:37.800502 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-0fdc892c-1d16-4271-9b7b-6ecfae514224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:37.810326 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:37.823057 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0614717c-15c4-4283-abe4-acb8ddf8a248 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:36:37.824693 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0614717c-15c4-4283-abe4-acb8ddf8a248 service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:36:37.846179 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 682/1374] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:36:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:36:37.905532 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 693/1375] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:36:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:36:38.191416 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614717c-15c4-4283-abe4-acb8ddf8a248 service cinder] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:36:38.192718 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-0614717c-15c4-4283-abe4-acb8ddf8a248 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:38.287795 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614717c-15c4-4283-abe4-acb8ddf8a248 service cinder] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:36:38.390061 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:38.402341 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184aaf8b-2492-446b-a987-e21910b66b72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:36:38.402945 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184aaf8b-2492-446b-a987-e21910b66b72 service nova] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:38.403223 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184aaf8b-2492-446b-a987-e21910b66b72 service nova] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:38.403758 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184aaf8b-2492-446b-a987-e21910b66b72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:38.405392 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-184aaf8b-2492-446b-a987-e21910b66b72 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:38.497030 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184aaf8b-2492-446b-a987-e21910b66b72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:36:38.498634 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184aaf8b-2492-446b-a987-e21910b66b72 service nova] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:36:38.517305 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 683/1376] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:36:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:36:40.418012 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e0231-b0d9-4404-89d2-e83628b97ab1 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:40.418950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e0231-b0d9-4404-89d2-e83628b97ab1 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:40.419295 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e0231-b0d9-4404-89d2-e83628b97ab1 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:40.420768 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 694/1377] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:36:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:36:40.426817 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38e89d-2b53-45b3-97ae-e772fad21c1f None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:40.427270 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38e89d-2b53-45b3-97ae-e772fad21c1f None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:40.427370 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38e89d-2b53-45b3-97ae-e772fad21c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:40.495753 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-ff38e89d-2b53-45b3-97ae-e772fad21c1f None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:36:40.497079 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-ff38e89d-2b53-45b3-97ae-e772fad21c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:40.556236 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:40.595427 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 684/1378] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:36:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:36:40.885762 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e0231-b0d9-4404-89d2-e83628b97ab1 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:36:40.887286 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-c73e0231-b0d9-4404-89d2-e83628b97ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:40.939372 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:41.032931 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e0231-b0d9-4404-89d2-e83628b97ab1 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:36:41.146072 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd41292c-2690-4db1-b308-56f3e59bb987 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:36:41.146768 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd41292c-2690-4db1-b308-56f3e59bb987 service nova] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:41.148009 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd41292c-2690-4db1-b308-56f3e59bb987 service nova] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:41.148009 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd41292c-2690-4db1-b308-56f3e59bb987 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:41.150301 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-fd41292c-2690-4db1-b308-56f3e59bb987 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:41.162780 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:41.264040 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd41292c-2690-4db1-b308-56f3e59bb987 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:36:41.266464 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd41292c-2690-4db1-b308-56f3e59bb987 service nova] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:36:41.292880 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 695/1379] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:36:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:36:43.227714 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25756fe7-64d2-439c-a6ba-fb85e7e82b98 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:43.228270 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25756fe7-64d2-439c-a6ba-fb85e7e82b98 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:43.228270 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25756fe7-64d2-439c-a6ba-fb85e7e82b98 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:43.229757 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 685/1380] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:36:43.234955 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f032ab15-3efb-4f2b-ac8f-f34bcbb1fece None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:43.235387 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f032ab15-3efb-4f2b-ac8f-f34bcbb1fece None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:43.235492 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f032ab15-3efb-4f2b-ac8f-f34bcbb1fece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:43.314921 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-f032ab15-3efb-4f2b-ac8f-f34bcbb1fece None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:36:43.316376 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-f032ab15-3efb-4f2b-ac8f-f34bcbb1fece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:43.412395 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:43.426451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 696/1381] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:36:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:36:43.718895 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25756fe7-64d2-439c-a6ba-fb85e7e82b98 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:36:43.720554 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-25756fe7-64d2-439c-a6ba-fb85e7e82b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:43.875317 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25756fe7-64d2-439c-a6ba-fb85e7e82b98 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:36:43.925631 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:43.996286 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a9ebf-d7dc-4496-b949-50ee4f74b4d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:36:43.997050 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a9ebf-d7dc-4496-b949-50ee4f74b4d2 service nova] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:43.997427 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a9ebf-d7dc-4496-b949-50ee4f74b4d2 service nova] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:43.997789 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a9ebf-d7dc-4496-b949-50ee4f74b4d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:44.000307 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-057a9ebf-d7dc-4496-b949-50ee4f74b4d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:44.103425 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a9ebf-d7dc-4496-b949-50ee4f74b4d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:36:44.105386 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a9ebf-d7dc-4496-b949-50ee4f74b4d2 service nova] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:36:44.126563 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 686/1382] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:36:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:36:46.414594 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac43fd-130f-485f-9660-1b8235c03624 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:46.415150 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac43fd-130f-485f-9660-1b8235c03624 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:46.415572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac43fd-130f-485f-9660-1b8235c03624 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:46.417325 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 697/1383] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:36:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:36:46.422731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf02c10-877b-4aba-9f5e-a0ddce47c353 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:46.423146 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf02c10-877b-4aba-9f5e-a0ddce47c353 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:46.423295 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf02c10-877b-4aba-9f5e-a0ddce47c353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:46.488993 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-acf02c10-877b-4aba-9f5e-a0ddce47c353 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:36:46.490145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-acf02c10-877b-4aba-9f5e-a0ddce47c353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:46.492260 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:46.589342 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 687/1384] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:36:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:36:46.873336 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac43fd-130f-485f-9660-1b8235c03624 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:36:46.875375 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-73ac43fd-130f-485f-9660-1b8235c03624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:47.021365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac43fd-130f-485f-9660-1b8235c03624 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:36:47.084862 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:47.138550 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b1365c-893c-4b25-bf93-b78eac2fb217 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:36:47.139082 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1365c-893c-4b25-bf93-b78eac2fb217 service nova] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:47.139365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1365c-893c-4b25-bf93-b78eac2fb217 service nova] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:47.139639 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1365c-893c-4b25-bf93-b78eac2fb217 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:47.141320 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-15b1365c-893c-4b25-bf93-b78eac2fb217 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:47.230773 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b1365c-893c-4b25-bf93-b78eac2fb217 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:36:47.232443 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b1365c-893c-4b25-bf93-b78eac2fb217 service nova] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:36:47.251026 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 698/1385] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:36:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:36:49.137047 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96164801-6796-4d74-879d-40b01ea4acec None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:49.137589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96164801-6796-4d74-879d-40b01ea4acec None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:49.137589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96164801-6796-4d74-879d-40b01ea4acec None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:49.138915 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 688/1386] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:36:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:36:49.144750 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203b053-f2f0-4e2f-b91d-80a9aabecd9c None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:49.145182 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203b053-f2f0-4e2f-b91d-80a9aabecd9c None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:49.145284 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203b053-f2f0-4e2f-b91d-80a9aabecd9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:49.211371 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-3203b053-f2f0-4e2f-b91d-80a9aabecd9c None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:36:49.212618 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-3203b053-f2f0-4e2f-b91d-80a9aabecd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:49.238456 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:49.325479 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 699/1387] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:36:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:36:49.613080 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96164801-6796-4d74-879d-40b01ea4acec None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:36:49.614587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-96164801-6796-4d74-879d-40b01ea4acec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:49.745251 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:49.763262 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96164801-6796-4d74-879d-40b01ea4acec None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:36:49.878875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b75548a-73e2-4b60-adae-b20127f666c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:36:49.879468 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75548a-73e2-4b60-adae-b20127f666c6 service nova] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:49.879745 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75548a-73e2-4b60-adae-b20127f666c6 service nova] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:49.880043 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75548a-73e2-4b60-adae-b20127f666c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:49.881677 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-4b75548a-73e2-4b60-adae-b20127f666c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:49.972894 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b75548a-73e2-4b60-adae-b20127f666c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:36:49.974509 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b75548a-73e2-4b60-adae-b20127f666c6 service nova] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:36:49.979613 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:49.994489 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 689/1388] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:36:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:36:51.932640 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d511f16-242c-4158-9632-b7c6bfca7197 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:51.933180 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d511f16-242c-4158-9632-b7c6bfca7197 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:51.933180 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d511f16-242c-4158-9632-b7c6bfca7197 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:51.934528 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 700/1389] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:36:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:36:51.940478 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069b2a5-9051-40c7-9dcf-2fb5a631f630 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:51.940914 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069b2a5-9051-40c7-9dcf-2fb5a631f630 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:51.941018 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069b2a5-9051-40c7-9dcf-2fb5a631f630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:52.010859 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-7069b2a5-9051-40c7-9dcf-2fb5a631f630 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:36:52.012245 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-7069b2a5-9051-40c7-9dcf-2fb5a631f630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:52.108208 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 690/1390] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:36:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:36:52.396748 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d511f16-242c-4158-9632-b7c6bfca7197 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:36:52.398335 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-6d511f16-242c-4158-9632-b7c6bfca7197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:52.560667 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d511f16-242c-4158-9632-b7c6bfca7197 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:36:52.570442 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:52.676058 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25806f0e-1570-48c0-8b7e-8d4ca0afb95d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:36:52.676588 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25806f0e-1570-48c0-8b7e-8d4ca0afb95d service nova] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:52.676866 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25806f0e-1570-48c0-8b7e-8d4ca0afb95d service nova] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:52.677135 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25806f0e-1570-48c0-8b7e-8d4ca0afb95d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:52.678715 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-25806f0e-1570-48c0-8b7e-8d4ca0afb95d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:52.771353 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25806f0e-1570-48c0-8b7e-8d4ca0afb95d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:36:52.773000 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25806f0e-1570-48c0-8b7e-8d4ca0afb95d service nova] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:36:52.791927 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 701/1391] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:36:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:36:52.807507 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ec07b-e603-4101-b8b6-390d6af543d9 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:52.807790 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ec07b-e603-4101-b8b6-390d6af543d9 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:52.808113 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ec07b-e603-4101-b8b6-390d6af543d9 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:52.809460 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 691/1392] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:36:52.814792 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf0c57-2278-447a-a187-1ca09abb9ccf None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:52.815077 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf0c57-2278-447a-a187-1ca09abb9ccf None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:52.815353 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf0c57-2278-447a-a187-1ca09abb9ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:52.831106 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:52.886809 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-4caf0c57-2278-447a-a187-1ca09abb9ccf None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:36:52.888097 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-4caf0c57-2278-447a-a187-1ca09abb9ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:52.992904 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 702/1393] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:36:53.290637 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b20d0a-44dc-4d07-bc0b-f437ae949f7e None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:53.291070 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b20d0a-44dc-4d07-bc0b-f437ae949f7e None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:53.291173 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b20d0a-44dc-4d07-bc0b-f437ae949f7e None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:53.292540 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 692/1394] 10.222.0.26 () {54 vars in 967 bytes} [Wed Aug 17 23:36:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:36:53.297569 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62425da-5908-480e-abd2-2e21380ee729 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:53.297860 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62425da-5908-480e-abd2-2e21380ee729 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:53.298235 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62425da-5908-480e-abd2-2e21380ee729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:53.371796 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-c62425da-5908-480e-abd2-2e21380ee729 None None] MFA Rules not processed for user `9c08c497a14c426f9cfe0dd8c8100a21`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:36:53.374113 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-c62425da-5908-480e-abd2-2e21380ee729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:53.409525 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:53.457232 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 703/1395] 10.222.0.26 () {58 vars in 1064 bytes} [Wed Aug 17 23:36:53 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:36:53.465010 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b20d0a-44dc-4d07-bc0b-f437ae949f7e None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:36:53.466594 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-39b20d0a-44dc-4d07-bc0b-f437ae949f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:53.539260 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:53.569492 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b20d0a-44dc-4d07-bc0b-f437ae949f7e None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:36:53.641131 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc81a1f-4488-4d4c-9a52-31e59571e238 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c08c497a14c426f9cfe0dd8c8100a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:36:53.641879 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc81a1f-4488-4d4c-9a52-31e59571e238 service glance] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:53.642163 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc81a1f-4488-4d4c-9a52-31e59571e238 service glance] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:53.642435 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc81a1f-4488-4d4c-9a52-31e59571e238 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:53.644059 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-ecc81a1f-4488-4d4c-9a52-31e59571e238 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:53.752418 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc81a1f-4488-4d4c-9a52-31e59571e238 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:36:53.753853 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc81a1f-4488-4d4c-9a52-31e59571e238 service glance] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:36:53.770114 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:53.773647 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 693/1396] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:36:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 23:36:54.497267 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff26ffc-74a8-440b-b169-368b8a82bd25 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:54.497721 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff26ffc-74a8-440b-b169-368b8a82bd25 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:54.497826 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff26ffc-74a8-440b-b169-368b8a82bd25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:54.567678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-6ff26ffc-74a8-440b-b169-368b8a82bd25 None None] MFA Rules not processed for user `d0a13dbffb4a49508b7fb661cf84e803`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:36:54.568968 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-6ff26ffc-74a8-440b-b169-368b8a82bd25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:54.661342 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 704/1397] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Aug 17 23:36:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:36:54.707041 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb699d89-60f8-4ab0-a0e9-0de6b9e0ea45 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:54.707539 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb699d89-60f8-4ab0-a0e9-0de6b9e0ea45 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:54.707664 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb699d89-60f8-4ab0-a0e9-0de6b9e0ea45 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:54.709160 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 694/1398] 10.222.0.26 () {54 vars in 943 bytes} [Wed Aug 17 23:36:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:36:54.715409 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e02560-c445-49c9-a1f1-ecebdde2dc1a None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:54.715694 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e02560-c445-49c9-a1f1-ecebdde2dc1a None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:54.715985 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e02560-c445-49c9-a1f1-ecebdde2dc1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:54.735199 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1140e03-a9de-4da3-a657-93a28a95c7bf None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:54.735591 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1140e03-a9de-4da3-a657-93a28a95c7bf None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:54.735867 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1140e03-a9de-4da3-a657-93a28a95c7bf None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:54.737225 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 695/1399] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:36:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 23:36:54.742258 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd98782-706b-4632-b65e-f8b9c6454475 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:54.742607 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd98782-706b-4632-b65e-f8b9c6454475 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:54.742911 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd98782-706b-4632-b65e-f8b9c6454475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:54.776193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:54.795466 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-89e02560-c445-49c9-a1f1-ecebdde2dc1a None None] MFA Rules not processed for user `ee422ea90557444fabdc9c292865b9b8`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:36:54.796756 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-89e02560-c445-49c9-a1f1-ecebdde2dc1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:54.809024 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-bbd98782-706b-4632-b65e-f8b9c6454475 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:36:54.810214 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-bbd98782-706b-4632-b65e-f8b9c6454475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:54.902048 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 705/1400] 10.222.0.26 () {58 vars in 1040 bytes} [Wed Aug 17 23:36:54 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:36:54.907275 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 696/1401] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:36:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:36:54.913155 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e02560-c445-49c9-a1f1-ecebdde2dc1a None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:36:54.922609 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e02560-c445-49c9-a1f1-ecebdde2dc1a None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:36:55.019093 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:55.025162 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b678ada-a483-49a4-b5e3-ec52a5aa773c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee422ea90557444fabdc9c292865b9b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:36:55.025764 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b678ada-a483-49a4-b5e3-ec52a5aa773c service swift] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:55.026040 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b678ada-a483-49a4-b5e3-ec52a5aa773c service swift] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:55.026354 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b678ada-a483-49a4-b5e3-ec52a5aa773c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:55.027920 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-7b678ada-a483-49a4-b5e3-ec52a5aa773c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:55.108507 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b678ada-a483-49a4-b5e3-ec52a5aa773c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:36:55.110147 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b678ada-a483-49a4-b5e3-ec52a5aa773c service swift] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:36:55.118187 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 706/1402] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Aug 17 23:36:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 23:36:55.197269 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd98782-706b-4632-b65e-f8b9c6454475 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:36:55.198725 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-bbd98782-706b-4632-b65e-f8b9c6454475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:55.301593 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:55.342213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd98782-706b-4632-b65e-f8b9c6454475 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:36:55.474074 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1da0e02-e2b8-4633-afd5-118ca57ed374 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:36:55.474967 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da0e02-e2b8-4633-afd5-118ca57ed374 service nova] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:55.475413 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da0e02-e2b8-4633-afd5-118ca57ed374 service nova] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:55.475684 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da0e02-e2b8-4633-afd5-118ca57ed374 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:55.477340 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-f1da0e02-e2b8-4633-afd5-118ca57ed374 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:55.560106 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:55.582427 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1da0e02-e2b8-4633-afd5-118ca57ed374 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:36:55.584565 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1da0e02-e2b8-4633-afd5-118ca57ed374 service nova] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:36:55.607002 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 697/1403] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:36:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:36:57.554649 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25727e4-3273-48dc-84a0-1c93c8f458e1 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:57.555174 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25727e4-3273-48dc-84a0-1c93c8f458e1 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:57.555174 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25727e4-3273-48dc-84a0-1c93c8f458e1 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:57.556798 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 707/1404] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:36:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:36:57.565992 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d5f26-a308-4938-8dfb-93ded7572df4 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:57.568005 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d5f26-a308-4938-8dfb-93ded7572df4 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:57.568005 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d5f26-a308-4938-8dfb-93ded7572df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:57.664882 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-ee4d5f26-a308-4938-8dfb-93ded7572df4 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:36:57.666112 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-ee4d5f26-a308-4938-8dfb-93ded7572df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:57.776556 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 698/1405] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:36:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:36:58.065015 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25727e4-3273-48dc-84a0-1c93c8f458e1 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:36:58.067026 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-d25727e4-3273-48dc-84a0-1c93c8f458e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:58.219947 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:58.232539 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25727e4-3273-48dc-84a0-1c93c8f458e1 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:36:58.360011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dacef1-2ea2-49ee-8021-780f6e728e14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:36:58.360543 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dacef1-2ea2-49ee-8021-780f6e728e14 service nova] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:36:58.360820 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dacef1-2ea2-49ee-8021-780f6e728e14 service nova] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:36:58.361093 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dacef1-2ea2-49ee-8021-780f6e728e14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:36:58.362707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-75dacef1-2ea2-49ee-8021-780f6e728e14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:36:58.469990 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dacef1-2ea2-49ee-8021-780f6e728e14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:36:58.471623 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dacef1-2ea2-49ee-8021-780f6e728e14 service nova] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:36:58.484035 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:36:58.494740 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 708/1406] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:36:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:37:00.459476 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fadd0b-923c-490e-8126-643168e6f21e None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:00.459950 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fadd0b-923c-490e-8126-643168e6f21e None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:00.460093 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fadd0b-923c-490e-8126-643168e6f21e None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:00.465143 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 699/1407] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:37:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:37:00.469460 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f1041-b8f2-4f18-b6bc-26e89c9c343d None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:00.469914 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f1041-b8f2-4f18-b6bc-26e89c9c343d None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:00.470212 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f1041-b8f2-4f18-b6bc-26e89c9c343d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:00.547259 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-404f1041-b8f2-4f18-b6bc-26e89c9c343d None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:37:00.548562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-404f1041-b8f2-4f18-b6bc-26e89c9c343d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:00.659911 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 709/1408] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:37:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:37:00.950647 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fadd0b-923c-490e-8126-643168e6f21e None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:37:00.950647 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-37fadd0b-923c-490e-8126-643168e6f21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:01.013511 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:01.110964 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fadd0b-923c-490e-8126-643168e6f21e None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:37:01.230230 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da14467-0536-4413-bb12-6b65fb2947af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:37:01.231118 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da14467-0536-4413-bb12-6b65fb2947af service nova] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:01.231559 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da14467-0536-4413-bb12-6b65fb2947af service nova] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:01.232046 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da14467-0536-4413-bb12-6b65fb2947af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:01.233910 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-0da14467-0536-4413-bb12-6b65fb2947af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:01.247202 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:01.335502 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da14467-0536-4413-bb12-6b65fb2947af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:37:01.337224 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da14467-0536-4413-bb12-6b65fb2947af service nova] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:37:01.356993 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 700/1409] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:37:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:37:10.657984 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=67 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:11.353946 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=39 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:17.220923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b6199d-d508-45f6-8a34-2e056368b96e None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:17.221488 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b6199d-d508-45f6-8a34-2e056368b96e None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:17.221488 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b6199d-d508-45f6-8a34-2e056368b96e None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:17.222852 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 710/1410] 10.222.0.26 () {54 vars in 918 bytes} [Wed Aug 17 23:37:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:37:17.227878 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2392746-763f-4354-88bd-ab7dc387034d None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:17.228325 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2392746-763f-4354-88bd-ab7dc387034d None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:17.228448 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2392746-763f-4354-88bd-ab7dc387034d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:17.302567 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-c2392746-763f-4354-88bd-ab7dc387034d None None] MFA Rules not processed for user `1fc57d911a56423e96c736bc96270ba8`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:37:17.304471 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-c2392746-763f-4354-88bd-ab7dc387034d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:17.412793 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 701/1411] 10.222.0.26 () {58 vars in 1015 bytes} [Wed Aug 17 23:37:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:37:17.426944 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41294b9-bfee-4c36-9ad8-bb29eb1e905f None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:17.427230 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41294b9-bfee-4c36-9ad8-bb29eb1e905f None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:17.427510 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41294b9-bfee-4c36-9ad8-bb29eb1e905f None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:17.428938 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 711/1412] 10.222.0.26 () {54 vars in 918 bytes} [Wed Aug 17 23:37:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:37:17.433479 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a5ae8-d78f-409e-8b36-0fdb0c7cae10 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:17.433762 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a5ae8-d78f-409e-8b36-0fdb0c7cae10 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:17.434033 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a5ae8-d78f-409e-8b36-0fdb0c7cae10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:17.508314 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:17.510859 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-ac5a5ae8-d78f-409e-8b36-0fdb0c7cae10 None None] MFA Rules not processed for user `ec1dcbe43b0749c3bcc4af9100d771d1`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:37:17.512599 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-ac5a5ae8-d78f-409e-8b36-0fdb0c7cae10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:17.625496 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 702/1413] 10.222.0.26 () {58 vars in 1015 bytes} [Wed Aug 17 23:37:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:37:19.984874 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be8af1-32e0-43ad-8e0c-ef89e097780e None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:19.985342 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be8af1-32e0-43ad-8e0c-ef89e097780e None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:19.985445 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be8af1-32e0-43ad-8e0c-ef89e097780e None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:19.986766 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 712/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 23:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:37:19.992176 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f042d-547c-4a57-81a6-63194167d128 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:19.992617 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f042d-547c-4a57-81a6-63194167d128 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:19.992760 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f042d-547c-4a57-81a6-63194167d128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:20.056764 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-505f042d-547c-4a57-81a6-63194167d128 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:37:20.057948 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-505f042d-547c-4a57-81a6-63194167d128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:20.115201 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:20.157615 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 703/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 23:37:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:37:20.447435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be8af1-32e0-43ad-8e0c-ef89e097780e None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:37:20.449010 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-16be8af1-32e0-43ad-8e0c-ef89e097780e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:20.594742 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be8af1-32e0-43ad-8e0c-ef89e097780e None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:37:20.670789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:20.710527 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6da1f7-6578-448a-90d8-36889778fb41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:37:20.711123 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6da1f7-6578-448a-90d8-36889778fb41 service nova] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:20.711396 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6da1f7-6578-448a-90d8-36889778fb41 service nova] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:20.711662 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6da1f7-6578-448a-90d8-36889778fb41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:20.713273 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-5e6da1f7-6578-448a-90d8-36889778fb41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:20.798923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6da1f7-6578-448a-90d8-36889778fb41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:37:20.800572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6da1f7-6578-448a-90d8-36889778fb41 service nova] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:37:20.818338 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 713/1416] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:37:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:37:21.702942 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9938be-7414-477b-a6e7-b62e59295913 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:21.703450 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9938be-7414-477b-a6e7-b62e59295913 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:21.703450 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9938be-7414-477b-a6e7-b62e59295913 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:21.704806 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 704/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 17 23:37:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:37:21.709839 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e77b3-9739-4134-ae00-1669846251fc None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:21.710244 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e77b3-9739-4134-ae00-1669846251fc None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:21.710381 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e77b3-9739-4134-ae00-1669846251fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:21.785042 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-4d0e77b3-9739-4134-ae00-1669846251fc None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:37:21.786253 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-4d0e77b3-9739-4134-ae00-1669846251fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:21.812214 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:21.880524 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 714/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 17 23:37:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:37:22.031837 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9938be-7414-477b-a6e7-b62e59295913 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:37:22.033391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-3e9938be-7414-477b-a6e7-b62e59295913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:22.174969 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9938be-7414-477b-a6e7-b62e59295913 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:37:22.214473 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:22.288847 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f5bc64-7cb8-4e06-b8c6-bed89173dfe9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:37:22.289361 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5bc64-7cb8-4e06-b8c6-bed89173dfe9 service nova] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:22.289635 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5bc64-7cb8-4e06-b8c6-bed89173dfe9 service nova] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:22.289899 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5bc64-7cb8-4e06-b8c6-bed89173dfe9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:22.291493 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-80f5bc64-7cb8-4e06-b8c6-bed89173dfe9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:22.378792 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f5bc64-7cb8-4e06-b8c6-bed89173dfe9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:37:22.380394 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f5bc64-7cb8-4e06-b8c6-bed89173dfe9 service nova] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:37:22.398524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 705/1419] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:37:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:37:23.267809 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116f0b7-1db3-45f0-917d-a5d57cdaa93f None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:23.268401 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116f0b7-1db3-45f0-917d-a5d57cdaa93f None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:23.268401 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116f0b7-1db3-45f0-917d-a5d57cdaa93f None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:23.269727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 715/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 17 23:37:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:37:23.274580 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6835f1-6b54-424d-8366-5167f85a9e46 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:23.275002 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6835f1-6b54-424d-8366-5167f85a9e46 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:23.275114 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6835f1-6b54-424d-8366-5167f85a9e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:23.328969 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:23.351184 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-2c6835f1-6b54-424d-8366-5167f85a9e46 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:37:23.352426 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-2c6835f1-6b54-424d-8366-5167f85a9e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:23.447996 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 706/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 17 23:37:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:37:23.599468 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0116f0b7-1db3-45f0-917d-a5d57cdaa93f None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:37:23.600947 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-0116f0b7-1db3-45f0-917d-a5d57cdaa93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:23.758699 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0116f0b7-1db3-45f0-917d-a5d57cdaa93f None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:37:23.770951 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:23.868583 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20de13e-a9f9-414b-9312-c74fd9e7a3b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:37:23.869114 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20de13e-a9f9-414b-9312-c74fd9e7a3b5 service nova] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:23.869393 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20de13e-a9f9-414b-9312-c74fd9e7a3b5 service nova] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:23.869663 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20de13e-a9f9-414b-9312-c74fd9e7a3b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:23.871241 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-f20de13e-a9f9-414b-9312-c74fd9e7a3b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:23.957738 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20de13e-a9f9-414b-9312-c74fd9e7a3b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:37:23.959327 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20de13e-a9f9-414b-9312-c74fd9e7a3b5 service nova] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:37:23.976841 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 716/1422] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:37:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:37:24.833004 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470bf6a-6b0b-4010-af27-6811a93c1512 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:24.833539 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470bf6a-6b0b-4010-af27-6811a93c1512 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:24.833539 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470bf6a-6b0b-4010-af27-6811a93c1512 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:24.834931 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 707/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 17 23:37:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:37:24.839849 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4b170-ef0a-48f8-9f35-46798d407752 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:24.840261 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4b170-ef0a-48f8-9f35-46798d407752 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:24.840435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4b170-ef0a-48f8-9f35-46798d407752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:24.860422 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:24.916448 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-d6c4b170-ef0a-48f8-9f35-46798d407752 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:37:24.917642 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-d6c4b170-ef0a-48f8-9f35-46798d407752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:25.011019 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 717/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 17 23:37:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:37:25.160668 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470bf6a-6b0b-4010-af27-6811a93c1512 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:37:25.162103 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-8470bf6a-6b0b-4010-af27-6811a93c1512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:25.269731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:25.305686 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470bf6a-6b0b-4010-af27-6811a93c1512 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:37:25.418133 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f3b10-3226-452d-a518-46ad1597daac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:37:25.418657 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f3b10-3226-452d-a518-46ad1597daac service nova] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:25.418935 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f3b10-3226-452d-a518-46ad1597daac service nova] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:25.419205 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f3b10-3226-452d-a518-46ad1597daac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:25.421344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-fe5f3b10-3226-452d-a518-46ad1597daac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:25.496914 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:25.511040 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f3b10-3226-452d-a518-46ad1597daac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:37:25.512671 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f3b10-3226-452d-a518-46ad1597daac service nova] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:37:25.530445 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 708/1425] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:37:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:37:26.384667 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667a3f0-2fd9-4eb5-be4f-0ba95322b62d None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:26.385208 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667a3f0-2fd9-4eb5-be4f-0ba95322b62d None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:26.386631 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667a3f0-2fd9-4eb5-be4f-0ba95322b62d None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:26.388039 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 718/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 17 23:37:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:37:26.393739 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790ec8a-9d29-49a8-8c3f-ab5c574bba63 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:26.394157 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790ec8a-9d29-49a8-8c3f-ab5c574bba63 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:26.394270 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790ec8a-9d29-49a8-8c3f-ab5c574bba63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:26.476081 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-7790ec8a-9d29-49a8-8c3f-ab5c574bba63 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:37:26.477313 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-7790ec8a-9d29-49a8-8c3f-ab5c574bba63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:26.574072 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 709/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 17 23:37:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:37:26.726421 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f667a3f0-2fd9-4eb5-be4f-0ba95322b62d None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:37:26.727968 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-f667a3f0-2fd9-4eb5-be4f-0ba95322b62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:26.814141 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:26.882594 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f667a3f0-2fd9-4eb5-be4f-0ba95322b62d None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:37:26.998144 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd74ea6-694f-4c15-9a64-0421ea51588a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:37:26.998666 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd74ea6-694f-4c15-9a64-0421ea51588a service nova] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:26.998961 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd74ea6-694f-4c15-9a64-0421ea51588a service nova] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:26.999762 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd74ea6-694f-4c15-9a64-0421ea51588a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:27.001457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-4cd74ea6-694f-4c15-9a64-0421ea51588a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:27.048221 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:27.088397 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd74ea6-694f-4c15-9a64-0421ea51588a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:37:27.090248 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd74ea6-694f-4c15-9a64-0421ea51588a service nova] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:37:27.107752 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 719/1428] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:37:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:37:27.900126 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5848a86-1140-4131-b322-de45b1544269 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:27.900664 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5848a86-1140-4131-b322-de45b1544269 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:27.900664 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5848a86-1140-4131-b322-de45b1544269 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:27.902029 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 710/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 17 23:37:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 23:37:27.907873 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd06ed0-81cc-4b84-b877-2cb8ccf98844 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:27.908356 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd06ed0-81cc-4b84-b877-2cb8ccf98844 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:27.908457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd06ed0-81cc-4b84-b877-2cb8ccf98844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:27.981778 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-6fd06ed0-81cc-4b84-b877-2cb8ccf98844 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:37:27.982978 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-6fd06ed0-81cc-4b84-b877-2cb8ccf98844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:28.076538 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 720/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 17 23:37:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:37:28.227555 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5848a86-1140-4131-b322-de45b1544269 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:37:28.229101 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-a5848a86-1140-4131-b322-de45b1544269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:28.277894 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:28.377106 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5848a86-1140-4131-b322-de45b1544269 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:37:28.487430 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adeacee-db9b-4949-a26b-34c5fe7e564b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:37:28.487987 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adeacee-db9b-4949-a26b-34c5fe7e564b service nova] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:28.488281 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adeacee-db9b-4949-a26b-34c5fe7e564b service nova] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:28.488553 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adeacee-db9b-4949-a26b-34c5fe7e564b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:28.490171 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-8adeacee-db9b-4949-a26b-34c5fe7e564b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:28.496967 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:28.579330 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adeacee-db9b-4949-a26b-34c5fe7e564b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:37:28.580964 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adeacee-db9b-4949-a26b-34c5fe7e564b service nova] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:37:28.599287 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 711/1431] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:37:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:37:29.394549 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598ad0a-8052-4212-a4fd-ebf8f7f0d311 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:29.395129 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598ad0a-8052-4212-a4fd-ebf8f7f0d311 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:29.395129 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598ad0a-8052-4212-a4fd-ebf8f7f0d311 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:29.396530 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 721/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 17 23:37:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:37:29.401548 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e06f12e-c9b4-4ed2-9830-2ddde93a6884 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:29.401942 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e06f12e-c9b4-4ed2-9830-2ddde93a6884 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:29.402094 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e06f12e-c9b4-4ed2-9830-2ddde93a6884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:29.477491 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-0e06f12e-c9b4-4ed2-9830-2ddde93a6884 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:37:29.478683 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-0e06f12e-c9b4-4ed2-9830-2ddde93a6884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:29.543875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:29.572199 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 712/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 17 23:37:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:37:29.720231 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598ad0a-8052-4212-a4fd-ebf8f7f0d311 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:37:29.721741 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-9598ad0a-8052-4212-a4fd-ebf8f7f0d311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:29.738784 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:29.866772 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598ad0a-8052-4212-a4fd-ebf8f7f0d311 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:37:29.954743 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:29.981260 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322800fa-abb9-4394-ba8f-0e0d0e60fcbe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:37:29.981798 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322800fa-abb9-4394-ba8f-0e0d0e60fcbe service nova] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:29.982095 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322800fa-abb9-4394-ba8f-0e0d0e60fcbe service nova] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:29.982350 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322800fa-abb9-4394-ba8f-0e0d0e60fcbe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:29.984499 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-322800fa-abb9-4394-ba8f-0e0d0e60fcbe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:30.071490 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322800fa-abb9-4394-ba8f-0e0d0e60fcbe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:37:30.073134 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322800fa-abb9-4394-ba8f-0e0d0e60fcbe service nova] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:37:30.090558 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 722/1434] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:37:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:37:30.894381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80397d27-df03-40eb-af80-3730d2136286 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:30.894932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80397d27-df03-40eb-af80-3730d2136286 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:30.894932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80397d27-df03-40eb-af80-3730d2136286 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:30.896302 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 713/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 17 23:37:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:37:30.901161 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b1ec4-4036-4e3e-a9ea-d9176a284807 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:30.901589 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b1ec4-4036-4e3e-a9ea-d9176a284807 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:30.901698 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b1ec4-4036-4e3e-a9ea-d9176a284807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:30.976460 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-3d3b1ec4-4036-4e3e-a9ea-d9176a284807 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:37:30.977658 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-3d3b1ec4-4036-4e3e-a9ea-d9176a284807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:31.018244 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:31.071885 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 723/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 17 23:37:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:37:31.220408 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80397d27-df03-40eb-af80-3730d2136286 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:37:31.221946 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-80397d27-df03-40eb-af80-3730d2136286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:31.364122 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80397d27-df03-40eb-af80-3730d2136286 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:37:31.417795 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:31.478629 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eaec2c-14b4-4c5c-8c8b-c70f27ade05d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:37:31.479168 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eaec2c-14b4-4c5c-8c8b-c70f27ade05d service nova] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:31.479442 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eaec2c-14b4-4c5c-8c8b-c70f27ade05d service nova] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:31.479711 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eaec2c-14b4-4c5c-8c8b-c70f27ade05d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:31.481315 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-66eaec2c-14b4-4c5c-8c8b-c70f27ade05d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:31.570532 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eaec2c-14b4-4c5c-8c8b-c70f27ade05d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:37:31.572167 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eaec2c-14b4-4c5c-8c8b-c70f27ade05d service nova] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:37:31.589502 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 714/1437] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:37:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:37:32.384616 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674176c7-eb92-4da2-a69f-f5bb40eb2469 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:32.385165 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674176c7-eb92-4da2-a69f-f5bb40eb2469 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:32.385165 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674176c7-eb92-4da2-a69f-f5bb40eb2469 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:32.386520 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 724/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 17 23:37:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:37:32.391986 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae9de5a-134f-43d4-9352-aa4db35a115c None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:32.392419 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae9de5a-134f-43d4-9352-aa4db35a115c None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:32.392530 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae9de5a-134f-43d4-9352-aa4db35a115c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:32.467143 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-eae9de5a-134f-43d4-9352-aa4db35a115c None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:37:32.468919 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-eae9de5a-134f-43d4-9352-aa4db35a115c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:32.470880 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:32.563397 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 715/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 17 23:37:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:37:32.718522 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674176c7-eb92-4da2-a69f-f5bb40eb2469 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:37:32.720045 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-674176c7-eb92-4da2-a69f-f5bb40eb2469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:32.863705 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674176c7-eb92-4da2-a69f-f5bb40eb2469 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:37:32.889771 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:32.974283 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163d961-46da-481d-afba-236b9f09ab8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:37:32.974888 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163d961-46da-481d-afba-236b9f09ab8b service nova] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:32.975165 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163d961-46da-481d-afba-236b9f09ab8b service nova] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:32.975438 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163d961-46da-481d-afba-236b9f09ab8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:32.977062 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-d163d961-46da-481d-afba-236b9f09ab8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:33.064710 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d163d961-46da-481d-afba-236b9f09ab8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:37:33.066298 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d163d961-46da-481d-afba-236b9f09ab8b service nova] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:37:33.083817 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 725/1440] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:37:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:37:34.863169 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb84767-915d-4009-af61-e2aa0ef7ac7f None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:34.863621 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb84767-915d-4009-af61-e2aa0ef7ac7f None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:34.863779 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb84767-915d-4009-af61-e2aa0ef7ac7f None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:34.865185 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 716/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 23:37:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:37:34.870451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152dd25-50bb-4af7-9daf-6685343405fa None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:34.870907 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152dd25-50bb-4af7-9daf-6685343405fa None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:34.871008 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152dd25-50bb-4af7-9daf-6685343405fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:34.912228 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:34.937720 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-9152dd25-50bb-4af7-9daf-6685343405fa None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:37:34.938956 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-9152dd25-50bb-4af7-9daf-6685343405fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:35.035309 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 726/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 23:37:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:37:35.317333 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb84767-915d-4009-af61-e2aa0ef7ac7f None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:37:35.318897 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-cbb84767-915d-4009-af61-e2aa0ef7ac7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:35.419456 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb84767-915d-4009-af61-e2aa0ef7ac7f None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:37:35.452272 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:35.499765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873e477-d120-4655-8245-f5b16764c9f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:37:35.500401 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873e477-d120-4655-8245-f5b16764c9f9 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:35.500711 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873e477-d120-4655-8245-f5b16764c9f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:35.500981 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873e477-d120-4655-8245-f5b16764c9f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:35.502580 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-0873e477-d120-4655-8245-f5b16764c9f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:35.725962 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0873e477-d120-4655-8245-f5b16764c9f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:37:35.727392 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0873e477-d120-4655-8245-f5b16764c9f9 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:37:35.745804 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 717/1443] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:37:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:37:37.782630 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb243b8-d9c0-45ad-85fb-cd7846ccbc62 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:37.783160 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb243b8-d9c0-45ad-85fb-cd7846ccbc62 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:37.783160 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb243b8-d9c0-45ad-85fb-cd7846ccbc62 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:37.784586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 727/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 23:37:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:37:37.789701 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f85c00-cd71-45ec-8a88-d2c5da414358 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:37.790109 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f85c00-cd71-45ec-8a88-d2c5da414358 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:37.790226 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f85c00-cd71-45ec-8a88-d2c5da414358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:37.854111 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-42f85c00-cd71-45ec-8a88-d2c5da414358 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:37:37.855314 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-42f85c00-cd71-45ec-8a88-d2c5da414358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:37.868334 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:37.950012 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 718/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 23:37:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:37:38.236490 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb243b8-d9c0-45ad-85fb-cd7846ccbc62 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:37:38.238017 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-bbb243b8-d9c0-45ad-85fb-cd7846ccbc62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:38.339727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb243b8-d9c0-45ad-85fb-cd7846ccbc62 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:37:38.352148 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:38.413440 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c1ab12-5056-4f02-a710-92d54c8adccc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:37:38.414072 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1ab12-5056-4f02-a710-92d54c8adccc service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:38.414347 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1ab12-5056-4f02-a710-92d54c8adccc service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:38.414616 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1ab12-5056-4f02-a710-92d54c8adccc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:38.416281 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-b0c1ab12-5056-4f02-a710-92d54c8adccc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:38.524369 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c1ab12-5056-4f02-a710-92d54c8adccc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:37:38.525742 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c1ab12-5056-4f02-a710-92d54c8adccc service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:37:38.543435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 728/1446] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:37:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:37:40.739690 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a47502-a545-4cb6-847e-e5037b4b26c6 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:40.740260 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a47502-a545-4cb6-847e-e5037b4b26c6 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:40.740260 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a47502-a545-4cb6-847e-e5037b4b26c6 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:40.741617 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 719/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 23:37:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:37:40.747136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a69db4-d0a9-429c-b25f-3d4a0f077a60 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:40.747543 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a69db4-d0a9-429c-b25f-3d4a0f077a60 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:40.747685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a69db4-d0a9-429c-b25f-3d4a0f077a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:40.794823 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:40.813119 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-42a69db4-d0a9-429c-b25f-3d4a0f077a60 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:37:40.814343 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-42a69db4-d0a9-429c-b25f-3d4a0f077a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:40.908440 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 729/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 23:37:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:37:41.189426 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a47502-a545-4cb6-847e-e5037b4b26c6 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:37:41.190973 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-a1a47502-a545-4cb6-847e-e5037b4b26c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:41.293195 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a47502-a545-4cb6-847e-e5037b4b26c6 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:37:41.332470 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:41.365999 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e604ee4-2dd6-4fee-ab4e-2fcd7900fd59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:37:41.366601 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e604ee4-2dd6-4fee-ab4e-2fcd7900fd59 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:41.366878 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e604ee4-2dd6-4fee-ab4e-2fcd7900fd59 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:41.367146 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e604ee4-2dd6-4fee-ab4e-2fcd7900fd59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:41.368840 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-7e604ee4-2dd6-4fee-ab4e-2fcd7900fd59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:41.478197 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e604ee4-2dd6-4fee-ab4e-2fcd7900fd59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:37:41.479645 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e604ee4-2dd6-4fee-ab4e-2fcd7900fd59 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:37:41.497898 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 720/1449] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:37:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:37:43.504241 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e22550-f2f9-4c02-b945-bee81474cc66 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:43.504744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e22550-f2f9-4c02-b945-bee81474cc66 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:43.504876 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e22550-f2f9-4c02-b945-bee81474cc66 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:43.506148 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 730/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 23:37:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:37:43.511515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4471b3-6f36-4022-a173-12234eb84da4 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:43.511893 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4471b3-6f36-4022-a173-12234eb84da4 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:43.512096 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4471b3-6f36-4022-a173-12234eb84da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:43.575864 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-fc4471b3-6f36-4022-a173-12234eb84da4 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:37:43.577126 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-fc4471b3-6f36-4022-a173-12234eb84da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:43.596680 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:43.672619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 721/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 23:37:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:37:43.954956 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e22550-f2f9-4c02-b945-bee81474cc66 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:37:43.956523 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-51e22550-f2f9-4c02-b945-bee81474cc66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:44.063949 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e22550-f2f9-4c02-b945-bee81474cc66 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:37:44.083748 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:44.136300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b3d7a6-be3a-419d-a5f6-1d2b178f022e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:37:44.136904 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b3d7a6-be3a-419d-a5f6-1d2b178f022e service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:44.137206 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b3d7a6-be3a-419d-a5f6-1d2b178f022e service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:44.137476 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b3d7a6-be3a-419d-a5f6-1d2b178f022e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:44.139057 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-f3b3d7a6-be3a-419d-a5f6-1d2b178f022e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:44.246129 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b3d7a6-be3a-419d-a5f6-1d2b178f022e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:37:44.247564 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b3d7a6-be3a-419d-a5f6-1d2b178f022e service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:37:44.265785 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 731/1452] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:37:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:37:46.206568 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dad327-9184-4eff-a6ec-c9c3f409817a None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:46.207103 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dad327-9184-4eff-a6ec-c9c3f409817a None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:46.207103 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dad327-9184-4eff-a6ec-c9c3f409817a None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:46.208480 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 722/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 23:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:37:46.213557 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8ba23-4fde-4fdc-8315-538826e56332 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:46.213949 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8ba23-4fde-4fdc-8315-538826e56332 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:46.214118 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8ba23-4fde-4fdc-8315-538826e56332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:46.276320 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:46.279140 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-6eb8ba23-4fde-4fdc-8315-538826e56332 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:37:46.281208 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-6eb8ba23-4fde-4fdc-8315-538826e56332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:46.375564 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 732/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 23:37:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:37:46.659882 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dad327-9184-4eff-a6ec-c9c3f409817a None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:37:46.661488 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-88dad327-9184-4eff-a6ec-c9c3f409817a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:46.765115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dad327-9184-4eff-a6ec-c9c3f409817a None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:37:46.812457 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:46.836872 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5869ae86-3d97-4d8c-b457-dcf5c469c008 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:37:46.837474 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869ae86-3d97-4d8c-b457-dcf5c469c008 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:46.837746 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869ae86-3d97-4d8c-b457-dcf5c469c008 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:46.838012 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869ae86-3d97-4d8c-b457-dcf5c469c008 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:46.839596 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-5869ae86-3d97-4d8c-b457-dcf5c469c008 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:46.945226 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5869ae86-3d97-4d8c-b457-dcf5c469c008 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:37:46.946586 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5869ae86-3d97-4d8c-b457-dcf5c469c008 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:37:46.965587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 723/1455] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:37:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:37:48.894987 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230ae3a-3e25-427c-856c-82220edc1915 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:48.895548 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230ae3a-3e25-427c-856c-82220edc1915 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:48.895548 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230ae3a-3e25-427c-856c-82220edc1915 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:48.896902 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 733/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 23:37:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:37:48.902572 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da134e1c-2423-4903-94d9-61b3774e51c6 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:48.903019 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da134e1c-2423-4903-94d9-61b3774e51c6 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:48.903130 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da134e1c-2423-4903-94d9-61b3774e51c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:48.967174 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-da134e1c-2423-4903-94d9-61b3774e51c6 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:37:48.969023 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-da134e1c-2423-4903-94d9-61b3774e51c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:48.996986 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:49.065618 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 724/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 23:37:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:37:49.350063 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4230ae3a-3e25-427c-856c-82220edc1915 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:37:49.351627 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-4230ae3a-3e25-427c-856c-82220edc1915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:49.455544 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4230ae3a-3e25-427c-856c-82220edc1915 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:37:49.483507 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:49.527749 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a3492-6d85-4b38-be58-0761ffb6f72e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:37:49.528389 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a3492-6d85-4b38-be58-0761ffb6f72e service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:49.528669 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a3492-6d85-4b38-be58-0761ffb6f72e service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:49.528942 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a3492-6d85-4b38-be58-0761ffb6f72e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:49.530825 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-911a3492-6d85-4b38-be58-0761ffb6f72e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:49.636794 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a3492-6d85-4b38-be58-0761ffb6f72e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:37:49.638185 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a3492-6d85-4b38-be58-0761ffb6f72e service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:37:49.655736 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 734/1458] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:37:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:37:51.685519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3afb94b-e11b-4269-b51c-969d0d81386f None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:51.686045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3afb94b-e11b-4269-b51c-969d0d81386f None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:51.686045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3afb94b-e11b-4269-b51c-969d0d81386f None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:51.687390 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 725/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 23:37:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:37:51.692488 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9defe29d-42fc-4210-a4b6-91d9670f3f6e None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:51.692920 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9defe29d-42fc-4210-a4b6-91d9670f3f6e None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:51.693033 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9defe29d-42fc-4210-a4b6-91d9670f3f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:51.758667 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-9defe29d-42fc-4210-a4b6-91d9670f3f6e None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:37:51.759881 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-9defe29d-42fc-4210-a4b6-91d9670f3f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:51.769003 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:51.855463 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 735/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 23:37:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:37:52.131439 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3afb94b-e11b-4269-b51c-969d0d81386f None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:37:52.133010 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-e3afb94b-e11b-4269-b51c-969d0d81386f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:52.234573 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3afb94b-e11b-4269-b51c-969d0d81386f None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:37:52.288446 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:52.306366 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c16cf3-68aa-40a1-bcd2-0dcc75f263eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:37:52.306963 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c16cf3-68aa-40a1-bcd2-0dcc75f263eb service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:52.307245 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c16cf3-68aa-40a1-bcd2-0dcc75f263eb service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:52.307515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c16cf3-68aa-40a1-bcd2-0dcc75f263eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:52.309145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-b8c16cf3-68aa-40a1-bcd2-0dcc75f263eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:52.416153 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c16cf3-68aa-40a1-bcd2-0dcc75f263eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:37:52.417521 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c16cf3-68aa-40a1-bcd2-0dcc75f263eb service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:37:52.436665 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 726/1461] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:37:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 23:37:54.442391 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fed7ba-90e9-4f91-9a8e-40ea1807db40 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:54.442947 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fed7ba-90e9-4f91-9a8e-40ea1807db40 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:54.442947 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fed7ba-90e9-4f91-9a8e-40ea1807db40 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:54.444364 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 736/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 23:37:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:37:54.449794 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183863f8-8110-41fd-9f77-d5dc2a9f07c3 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:54.450217 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183863f8-8110-41fd-9f77-d5dc2a9f07c3 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:54.450317 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183863f8-8110-41fd-9f77-d5dc2a9f07c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:54.515194 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-183863f8-8110-41fd-9f77-d5dc2a9f07c3 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:37:54.516411 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-183863f8-8110-41fd-9f77-d5dc2a9f07c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:54.555568 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:54.613858 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 727/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 23:37:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:37:54.892903 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fed7ba-90e9-4f91-9a8e-40ea1807db40 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:37:54.894449 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-d2fed7ba-90e9-4f91-9a8e-40ea1807db40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:54.996703 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fed7ba-90e9-4f91-9a8e-40ea1807db40 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:37:55.034462 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:55.069666 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03333503-639f-4def-b217-ae2f77e612bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:37:55.070288 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03333503-639f-4def-b217-ae2f77e612bb service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:55.070552 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03333503-639f-4def-b217-ae2f77e612bb service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:55.070822 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03333503-639f-4def-b217-ae2f77e612bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:55.072464 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-03333503-639f-4def-b217-ae2f77e612bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:55.179604 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03333503-639f-4def-b217-ae2f77e612bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:37:55.181004 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03333503-639f-4def-b217-ae2f77e612bb service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:37:55.199119 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 737/1464] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:37:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:37:57.100269 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a69ed-bbaf-493a-8c74-5a7aa01fa67f None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:57.100831 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a69ed-bbaf-493a-8c74-5a7aa01fa67f None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:57.100831 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a69ed-bbaf-493a-8c74-5a7aa01fa67f None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:57.102154 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 728/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 23:37:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:37:57.107392 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f32fb-f9a4-44f9-af67-37a17e61b8d7 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:57.108362 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f32fb-f9a4-44f9-af67-37a17e61b8d7 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:57.108489 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f32fb-f9a4-44f9-af67-37a17e61b8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:57.172803 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-db0f32fb-f9a4-44f9-af67-37a17e61b8d7 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:37:57.174005 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-db0f32fb-f9a4-44f9-af67-37a17e61b8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:57.192159 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:57.269972 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 738/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 23:37:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:37:57.550393 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004a69ed-bbaf-493a-8c74-5a7aa01fa67f None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:37:57.551987 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-004a69ed-bbaf-493a-8c74-5a7aa01fa67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:57.652230 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004a69ed-bbaf-493a-8c74-5a7aa01fa67f None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:37:57.718894 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:57.726197 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c708c5e-d017-4d4a-a794-0c134f5e9cc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:37:57.726804 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c708c5e-d017-4d4a-a794-0c134f5e9cc8 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:57.727081 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c708c5e-d017-4d4a-a794-0c134f5e9cc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:57.727350 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c708c5e-d017-4d4a-a794-0c134f5e9cc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:57.728984 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-2c708c5e-d017-4d4a-a794-0c134f5e9cc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:57.834507 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c708c5e-d017-4d4a-a794-0c134f5e9cc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:37:57.835950 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c708c5e-d017-4d4a-a794-0c134f5e9cc8 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:37:57.854703 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 729/1467] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:37:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:37:59.759082 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595639e-8a9e-4276-884c-7c98348895cc None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:59.759679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595639e-8a9e-4276-884c-7c98348895cc None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:59.759679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595639e-8a9e-4276-884c-7c98348895cc None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:59.761009 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 739/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 23:37:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:37:59.766223 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71454b0-acc6-4459-8f6c-ac1135a18594 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:37:59.766636 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71454b0-acc6-4459-8f6c-ac1135a18594 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:37:59.766771 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71454b0-acc6-4459-8f6c-ac1135a18594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:37:59.833750 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-b71454b0-acc6-4459-8f6c-ac1135a18594 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:37:59.834953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-b71454b0-acc6-4459-8f6c-ac1135a18594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:37:59.881069 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:37:59.930271 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 730/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 23:37:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:38:00.210159 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595639e-8a9e-4276-884c-7c98348895cc None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:38:00.211714 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-0595639e-8a9e-4276-884c-7c98348895cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:38:00.313521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595639e-8a9e-4276-884c-7c98348895cc None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:38:00.362298 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:38:00.388399 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c0046-d565-4db8-8c50-0408e071259e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:38:00.389631 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c0046-d565-4db8-8c50-0408e071259e service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:38:00.389915 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c0046-d565-4db8-8c50-0408e071259e service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:38:00.390183 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c0046-d565-4db8-8c50-0408e071259e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:38:00.391815 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-f97c0046-d565-4db8-8c50-0408e071259e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:38:00.496697 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c0046-d565-4db8-8c50-0408e071259e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:38:00.498128 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c0046-d565-4db8-8c50-0408e071259e service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:38:00.516729 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 740/1470] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:38:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:38:09.928504 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=22 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:38:10.513722 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=66 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:38:19.788168 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d696cc3-3299-469d-b81d-666aa9bc3562 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:38:19.788723 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d696cc3-3299-469d-b81d-666aa9bc3562 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:38:19.788723 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d696cc3-3299-469d-b81d-666aa9bc3562 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:38:19.790047 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 731/1471] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:38:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:38:19.795921 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45837a82-a269-4a79-a27d-35a26bd80de9 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:38:19.796480 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45837a82-a269-4a79-a27d-35a26bd80de9 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:38:19.796584 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45837a82-a269-4a79-a27d-35a26bd80de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:38:19.860968 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-45837a82-a269-4a79-a27d-35a26bd80de9 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:38:19.862177 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-45837a82-a269-4a79-a27d-35a26bd80de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:38:19.955876 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 741/1472] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:38:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:38:20.251869 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c072a3-8445-4c29-bcd7-2ba94c15e2ba None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:38:20.252187 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c072a3-8445-4c29-bcd7-2ba94c15e2ba None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:38:20.252459 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c072a3-8445-4c29-bcd7-2ba94c15e2ba None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:38:20.253815 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 732/1473] 10.222.0.26 () {54 vars in 967 bytes} [Wed Aug 17 23:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:38:20.258650 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556878c7-44ce-4557-9c51-6fa88bb82002 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:38:20.258938 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556878c7-44ce-4557-9c51-6fa88bb82002 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:38:20.259218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556878c7-44ce-4557-9c51-6fa88bb82002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:38:20.336253 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-556878c7-44ce-4557-9c51-6fa88bb82002 None None] MFA Rules not processed for user `9c08c497a14c426f9cfe0dd8c8100a21`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:38:20.337569 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-556878c7-44ce-4557-9c51-6fa88bb82002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:38:20.359334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:38:20.414256 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 742/1474] 10.222.0.26 () {58 vars in 1064 bytes} [Wed Aug 17 23:38:20 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:38:20.421792 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c072a3-8445-4c29-bcd7-2ba94c15e2ba None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:38:20.423455 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-25c072a3-8445-4c29-bcd7-2ba94c15e2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:38:20.525113 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c072a3-8445-4c29-bcd7-2ba94c15e2ba None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:38:20.596117 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29abe927-73fa-4712-a1d9-3c538f381c01 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c08c497a14c426f9cfe0dd8c8100a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:38:20.596720 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abe927-73fa-4712-a1d9-3c538f381c01 service glance] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:38:20.596996 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abe927-73fa-4712-a1d9-3c538f381c01 service glance] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:38:20.597262 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abe927-73fa-4712-a1d9-3c538f381c01 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:38:20.598848 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-29abe927-73fa-4712-a1d9-3c538f381c01 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:38:20.650947 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:38:20.705843 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29abe927-73fa-4712-a1d9-3c538f381c01 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:38:20.707287 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29abe927-73fa-4712-a1d9-3c538f381c01 service glance] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:38:20.725962 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 733/1475] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:38:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:38:22.687713 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94caa5b8-961b-4a81-b76d-349de8d40ee8 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:38:22.688260 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94caa5b8-961b-4a81-b76d-349de8d40ee8 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:38:22.688260 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94caa5b8-961b-4a81-b76d-349de8d40ee8 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:38:22.689616 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 743/1476] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:38:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:38:22.694634 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1777e6-eadf-40a9-8721-858a9f58afbf None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:38:22.695059 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1777e6-eadf-40a9-8721-858a9f58afbf None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:38:22.695221 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1777e6-eadf-40a9-8721-858a9f58afbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:38:22.759193 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-2b1777e6-eadf-40a9-8721-858a9f58afbf None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:38:22.760403 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-2b1777e6-eadf-40a9-8721-858a9f58afbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:38:22.854349 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 734/1477] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:38:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:38:23.124606 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94caa5b8-961b-4a81-b76d-349de8d40ee8 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:38:23.126110 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-94caa5b8-961b-4a81-b76d-349de8d40ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:38:23.247786 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94caa5b8-961b-4a81-b76d-349de8d40ee8 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:38:23.295446 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:38:23.339637 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92893a3f-19c8-4493-b0b1-b4636d85796c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:38:23.340265 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92893a3f-19c8-4493-b0b1-b4636d85796c admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:38:23.340545 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92893a3f-19c8-4493-b0b1-b4636d85796c admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:38:23.340816 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92893a3f-19c8-4493-b0b1-b4636d85796c admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:38:23.341449 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92893a3f-19c8-4493-b0b1-b4636d85796c admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:38:23.343233 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: WARNING py.warnings [None req-92893a3f-19c8-4493-b0b1-b4636d85796c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 17 23:38:23.343233 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: warnings.warn(msg) Aug 17 23:38:23.343233 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: Aug 17 23:38:23.344417 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92893a3f-19c8-4493-b0b1-b4636d85796c admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:38:23.362642 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 744/1478] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Aug 17 23:38:23 2022] GET /identity/v3/projects => generated 2553 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:38:25.304456 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9c5b4-f313-4f5a-ad77-863971bbb696 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:38:25.304989 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9c5b4-f313-4f5a-ad77-863971bbb696 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:38:25.304989 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9c5b4-f313-4f5a-ad77-863971bbb696 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:38:25.306350 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 735/1479] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:38:25.312336 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ebc41-0d85-40fc-9edf-762702927080 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:38:25.312734 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ebc41-0d85-40fc-9edf-762702927080 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:38:25.312914 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ebc41-0d85-40fc-9edf-762702927080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:38:25.377795 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-569ebc41-0d85-40fc-9edf-762702927080 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:38:25.379012 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-569ebc41-0d85-40fc-9edf-762702927080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:38:25.477272 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 745/1480] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:38:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:38:25.771431 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9c5b4-f313-4f5a-ad77-863971bbb696 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:38:25.772997 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-a2f9c5b4-f313-4f5a-ad77-863971bbb696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:38:25.782960 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:38:25.915654 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9c5b4-f313-4f5a-ad77-863971bbb696 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:38:25.993317 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:38:26.026029 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b856ec58-f9e3-4603-9667-7af9fb2ad4e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:38:26.026569 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856ec58-f9e3-4603-9667-7af9fb2ad4e9 service nova] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:38:26.026844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856ec58-f9e3-4603-9667-7af9fb2ad4e9 service nova] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:38:26.027112 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856ec58-f9e3-4603-9667-7af9fb2ad4e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:38:26.028748 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-b856ec58-f9e3-4603-9667-7af9fb2ad4e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:38:26.116177 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b856ec58-f9e3-4603-9667-7af9fb2ad4e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:38:26.117772 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b856ec58-f9e3-4603-9667-7af9fb2ad4e9 service nova] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:38:26.135168 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 736/1481] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:38:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:38:28.259888 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1eecda-32c4-4e2b-92bf-420b380b6868 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:38:28.260507 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1eecda-32c4-4e2b-92bf-420b380b6868 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:38:28.260507 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1eecda-32c4-4e2b-92bf-420b380b6868 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:38:28.261787 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 746/1482] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:38:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:38:28.267632 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197dd621-7f6e-4d1e-b52a-2c656e79f717 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:38:28.268102 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197dd621-7f6e-4d1e-b52a-2c656e79f717 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:38:28.268215 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197dd621-7f6e-4d1e-b52a-2c656e79f717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:38:28.332638 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-197dd621-7f6e-4d1e-b52a-2c656e79f717 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:38:28.333846 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-197dd621-7f6e-4d1e-b52a-2c656e79f717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:38:28.379065 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:38:28.429422 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 737/1483] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:38:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:38:28.711900 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1eecda-32c4-4e2b-92bf-420b380b6868 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:38:28.713460 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-db1eecda-32c4-4e2b-92bf-420b380b6868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:38:28.736567 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:38:28.934655 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1eecda-32c4-4e2b-92bf-420b380b6868 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:38:29.005572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c478f3c-72d0-4be7-892e-8e0c2b5e968d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:38:29.006162 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c478f3c-72d0-4be7-892e-8e0c2b5e968d service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:38:29.006434 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c478f3c-72d0-4be7-892e-8e0c2b5e968d service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:38:29.006701 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c478f3c-72d0-4be7-892e-8e0c2b5e968d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:38:29.008413 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-9c478f3c-72d0-4be7-892e-8e0c2b5e968d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:38:29.081860 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:38:29.117354 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c478f3c-72d0-4be7-892e-8e0c2b5e968d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:38:29.118788 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c478f3c-72d0-4be7-892e-8e0c2b5e968d service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:38:29.136427 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 747/1484] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:38:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:38:31.061747 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7306b-0c93-4993-9a41-ba9b9691c0f5 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:38:31.062360 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7306b-0c93-4993-9a41-ba9b9691c0f5 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:38:31.062360 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7306b-0c93-4993-9a41-ba9b9691c0f5 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:38:31.063641 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 738/1485] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:38:31.069165 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b58aca-9b28-42d5-a3d7-8f2362ba6285 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:38:31.069575 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b58aca-9b28-42d5-a3d7-8f2362ba6285 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:38:31.069708 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b58aca-9b28-42d5-a3d7-8f2362ba6285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:38:31.133067 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-87b58aca-9b28-42d5-a3d7-8f2362ba6285 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:38:31.134254 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-87b58aca-9b28-42d5-a3d7-8f2362ba6285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:38:31.225838 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 748/1486] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:38:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:38:31.509084 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7306b-0c93-4993-9a41-ba9b9691c0f5 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:38:31.510693 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-56b7306b-0c93-4993-9a41-ba9b9691c0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:38:31.611159 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7306b-0c93-4993-9a41-ba9b9691c0f5 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:38:31.681068 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:38:31.683046 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d536ad-c1d1-49a6-8ce6-cdea138b49de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:38:31.683908 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d536ad-c1d1-49a6-8ce6-cdea138b49de service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:38:31.684275 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d536ad-c1d1-49a6-8ce6-cdea138b49de service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:38:31.684754 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d536ad-c1d1-49a6-8ce6-cdea138b49de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:38:31.687099 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-23d536ad-c1d1-49a6-8ce6-cdea138b49de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:38:31.793588 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d536ad-c1d1-49a6-8ce6-cdea138b49de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:38:31.795040 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d536ad-c1d1-49a6-8ce6-cdea138b49de service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:38:31.814168 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 739/1487] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:38:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:38:33.777029 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c7b6f-cd80-420c-8eb8-f4b5fdb9907d None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:38:33.777609 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c7b6f-cd80-420c-8eb8-f4b5fdb9907d None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:38:33.777609 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c7b6f-cd80-420c-8eb8-f4b5fdb9907d None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:38:33.778966 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 749/1488] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:38:33.784283 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88326b5b-395c-494b-8a2f-45ed08d51b3a None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:38:33.784705 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88326b5b-395c-494b-8a2f-45ed08d51b3a None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:38:33.784825 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88326b5b-395c-494b-8a2f-45ed08d51b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:38:33.848839 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-88326b5b-395c-494b-8a2f-45ed08d51b3a None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:38:33.850048 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-88326b5b-395c-494b-8a2f-45ed08d51b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:38:33.901065 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:38:33.944834 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 740/1489] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:38:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:38:34.234182 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c7b6f-cd80-420c-8eb8-f4b5fdb9907d None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:38:34.235733 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-531c7b6f-cd80-420c-8eb8-f4b5fdb9907d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:38:34.272035 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:38:34.338338 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c7b6f-cd80-420c-8eb8-f4b5fdb9907d None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:38:34.409469 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de641a6-8552-4d24-84a6-ceb155476838 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:38:34.410069 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de641a6-8552-4d24-84a6-ceb155476838 service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:38:34.410345 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de641a6-8552-4d24-84a6-ceb155476838 service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:38:34.410616 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de641a6-8552-4d24-84a6-ceb155476838 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:38:34.412221 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-6de641a6-8552-4d24-84a6-ceb155476838 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:38:34.490880 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:38:34.518785 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de641a6-8552-4d24-84a6-ceb155476838 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:38:34.520378 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de641a6-8552-4d24-84a6-ceb155476838 service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:38:34.539476 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 750/1490] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:38:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:38:36.587193 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949a93b-3995-42ea-87da-ab37af6eccc1 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:38:36.587736 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949a93b-3995-42ea-87da-ab37af6eccc1 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:38:36.587736 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949a93b-3995-42ea-87da-ab37af6eccc1 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:38:36.589138 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 741/1491] 10.222.0.26 () {54 vars in 927 bytes} [Wed Aug 17 23:38:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:38:36.595193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74736650-e51d-42a7-8b9a-ac7dff6cd175 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:38:36.595642 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74736650-e51d-42a7-8b9a-ac7dff6cd175 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:38:36.595746 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74736650-e51d-42a7-8b9a-ac7dff6cd175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:38:36.659505 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-74736650-e51d-42a7-8b9a-ac7dff6cd175 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:38:36.660763 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-74736650-e51d-42a7-8b9a-ac7dff6cd175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:38:36.754812 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 751/1492] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Aug 17 23:38:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:38:37.044237 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c949a93b-3995-42ea-87da-ab37af6eccc1 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:38:37.046303 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-c949a93b-3995-42ea-87da-ab37af6eccc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:38:37.147112 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c949a93b-3995-42ea-87da-ab37af6eccc1 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:38:37.217291 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8013cb1a-cfc0-4d99-9883-1f0b7d055f6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:38:37.217878 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013cb1a-cfc0-4d99-9883-1f0b7d055f6f service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:38:37.218147 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013cb1a-cfc0-4d99-9883-1f0b7d055f6f service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:38:37.218407 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013cb1a-cfc0-4d99-9883-1f0b7d055f6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:38:37.219995 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-8013cb1a-cfc0-4d99-9883-1f0b7d055f6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:38:37.228726 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:38:37.327587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8013cb1a-cfc0-4d99-9883-1f0b7d055f6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:38:37.328975 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8013cb1a-cfc0-4d99-9883-1f0b7d055f6f service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:38:37.347467 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 742/1493] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:38:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:38:46.752682 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=82 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:38:47.345143 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=50 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:39:47.709572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7eaa80-c373-438b-93a2-aad794a78173 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:39:47.710120 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7eaa80-c373-438b-93a2-aad794a78173 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:39:47.710120 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7eaa80-c373-438b-93a2-aad794a78173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:39:47.785557 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-2d7eaa80-c373-438b-93a2-aad794a78173 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:39:47.786756 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-2d7eaa80-c373-438b-93a2-aad794a78173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:39:47.879400 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 752/1494] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:39:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:39:47.887374 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f37cc-aec2-4cda-bc57-6c9b419267a3 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:39:47.887848 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f37cc-aec2-4cda-bc57-6c9b419267a3 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:39:47.887954 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f37cc-aec2-4cda-bc57-6c9b419267a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:39:47.961859 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-317f37cc-aec2-4cda-bc57-6c9b419267a3 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:39:47.962971 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-317f37cc-aec2-4cda-bc57-6c9b419267a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:39:48.059240 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 743/1495] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:39:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:39:48.064430 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7eaa80-c373-438b-93a2-aad794a78173 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:39:48.065627 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-2d7eaa80-c373-438b-93a2-aad794a78173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:39:48.077473 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7eaa80-c373-438b-93a2-aad794a78173 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:39:48.137104 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:39:48.171627 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d630f5-075a-4765-bcd3-0b92d9953aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:39:48.172258 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d630f5-075a-4765-bcd3-0b92d9953aa4 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:39:48.172544 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d630f5-075a-4765-bcd3-0b92d9953aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:39:48.172811 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d630f5-075a-4765-bcd3-0b92d9953aa4 admin admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:39:48.173447 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d630f5-075a-4765-bcd3-0b92d9953aa4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:39:48.173748 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d630f5-075a-4765-bcd3-0b92d9953aa4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:39:48.174751 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: WARNING py.warnings [None req-62d630f5-075a-4765-bcd3-0b92d9953aa4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 17 23:39:48.174751 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: warnings.warn(msg) Aug 17 23:39:48.174751 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: Aug 17 23:39:48.175629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d630f5-075a-4765-bcd3-0b92d9953aa4 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:39:48.187135 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 753/1496] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:39:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:39:48.195400 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f37cc-aec2-4cda-bc57-6c9b419267a3 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:39:48.205209 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f37cc-aec2-4cda-bc57-6c9b419267a3 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:39:48.265600 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:39:48.300054 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c425e5c-185a-4a22-a2cf-fbe1115dd48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:39:48.300651 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c425e5c-185a-4a22-a2cf-fbe1115dd48a admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:39:48.300934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c425e5c-185a-4a22-a2cf-fbe1115dd48a admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:39:48.301228 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c425e5c-185a-4a22-a2cf-fbe1115dd48a admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:39:48.301854 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c425e5c-185a-4a22-a2cf-fbe1115dd48a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:39:48.303577 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: WARNING py.warnings [None req-8c425e5c-185a-4a22-a2cf-fbe1115dd48a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 17 23:39:48.303577 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: warnings.warn(msg) Aug 17 23:39:48.303577 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: Aug 17 23:39:48.304793 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c425e5c-185a-4a22-a2cf-fbe1115dd48a admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:39:48.359473 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 744/1497] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:39:48 2022] POST /identity/v3/projects => generated 379 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:39:48.363634 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d630f5-075a-4765-bcd3-0b92d9953aa4 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:39:48.365225 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-62d630f5-075a-4765-bcd3-0b92d9953aa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:39:48.397858 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d630f5-075a-4765-bcd3-0b92d9953aa4 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:39:48.490203 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8f143-f43c-48f7-92a5-3f7e70abc8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:39:48.490801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8f143-f43c-48f7-92a5-3f7e70abc8a8 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:39:48.491071 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8f143-f43c-48f7-92a5-3f7e70abc8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:39:48.491339 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8f143-f43c-48f7-92a5-3f7e70abc8a8 admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:39:48.491952 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd8f143-f43c-48f7-92a5-3f7e70abc8a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:39:48.492975 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: WARNING py.warnings [None req-0dd8f143-f43c-48f7-92a5-3f7e70abc8a8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 17 23:39:48.492975 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: warnings.warn(msg) Aug 17 23:39:48.492975 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: Aug 17 23:39:48.494174 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd8f143-f43c-48f7-92a5-3f7e70abc8a8 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:39:48.535439 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 754/1498] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:39:48 2022] POST /identity/v3/users => generated 290 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:39:48.539466 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c425e5c-185a-4a22-a2cf-fbe1115dd48a admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:39:48.541074 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-8c425e5c-185a-4a22-a2cf-fbe1115dd48a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:39:48.601858 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c425e5c-185a-4a22-a2cf-fbe1115dd48a admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:39:48.684208 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:39:48.695905 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6333b44-f45a-4f0a-b659-c0c228cf94c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:39:48.696537 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6333b44-f45a-4f0a-b659-c0c228cf94c2 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:39:48.696816 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6333b44-f45a-4f0a-b659-c0c228cf94c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:39:48.697089 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6333b44-f45a-4f0a-b659-c0c228cf94c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:39:48.697729 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6333b44-f45a-4f0a-b659-c0c228cf94c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:39:48.698777 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: WARNING py.warnings [None req-f6333b44-f45a-4f0a-b659-c0c228cf94c2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 17 23:39:48.698777 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: warnings.warn(msg) Aug 17 23:39:48.698777 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: Aug 17 23:39:48.699668 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6333b44-f45a-4f0a-b659-c0c228cf94c2 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:39:48.711866 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 745/1499] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:39:48 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:39:48.715945 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8f143-f43c-48f7-92a5-3f7e70abc8a8 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:39:48.717957 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-0dd8f143-f43c-48f7-92a5-3f7e70abc8a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:39:48.729354 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8f143-f43c-48f7-92a5-3f7e70abc8a8 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:39:48.754101 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:39:48.824732 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b5dc23-f483-4817-8dc4-3a58564dbfab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:39:48.825260 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b5dc23-f483-4817-8dc4-3a58564dbfab admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:39:48.825534 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b5dc23-f483-4817-8dc4-3a58564dbfab admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:39:48.825798 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b5dc23-f483-4817-8dc4-3a58564dbfab admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:39:48.826424 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b5dc23-f483-4817-8dc4-3a58564dbfab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:39:48.827453 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: WARNING py.warnings [None req-e6b5dc23-f483-4817-8dc4-3a58564dbfab admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 17 23:39:48.827453 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: warnings.warn(msg) Aug 17 23:39:48.827453 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: Aug 17 23:39:48.828359 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b5dc23-f483-4817-8dc4-3a58564dbfab admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:39:48.840181 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 755/1500] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:39:48 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:39:48.844819 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6333b44-f45a-4f0a-b659-c0c228cf94c2 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:39:48.846395 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-f6333b44-f45a-4f0a-b659-c0c228cf94c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:39:48.879236 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6333b44-f45a-4f0a-b659-c0c228cf94c2 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:39:48.971973 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b2209-d257-4028-86c5-69eeb25eae0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:39:48.972678 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b2209-d257-4028-86c5-69eeb25eae0a admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:39:48.972903 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b2209-d257-4028-86c5-69eeb25eae0a admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:39:48.973176 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b2209-d257-4028-86c5-69eeb25eae0a admin admin] PATH_INFO: `/v3/projects/9892eb3314e645d0a2d78266d38323cb/users/4d9d381b253647dd8e312d6ac2121c35/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:39:48.994688 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7b2209-d257-4028-86c5-69eeb25eae0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9892eb3314e645d0a2d78266d38323cb, user_id=4d9d381b253647dd8e312d6ac2121c35, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:39:48.995715 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: WARNING py.warnings [None req-7d7b2209-d257-4028-86c5-69eeb25eae0a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 17 23:39:48.995715 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: warnings.warn(msg) Aug 17 23:39:48.995715 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: Aug 17 23:39:48.997680 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7b2209-d257-4028-86c5-69eeb25eae0a admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:39:49.022661 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 746/1501] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:39:48 2022] PUT /identity/v3/projects/9892eb3314e645d0a2d78266d38323cb/users/4d9d381b253647dd8e312d6ac2121c35/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:39:49.035603 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299dc6c-9a87-4368-9b6a-a5c4ae978b24 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:39:49.035891 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299dc6c-9a87-4368-9b6a-a5c4ae978b24 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:39:49.036200 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299dc6c-9a87-4368-9b6a-a5c4ae978b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:39:49.092612 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-8299dc6c-9a87-4368-9b6a-a5c4ae978b24 None None] MFA Rules not processed for user `4d9d381b253647dd8e312d6ac2121c35`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:39:49.093739 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-8299dc6c-9a87-4368-9b6a-a5c4ae978b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:39:49.168656 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 756/1502] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:39:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:39:49.182161 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5a374-40db-4b32-b54b-f7e813631b3b None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:39:49.182448 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5a374-40db-4b32-b54b-f7e813631b3b None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:39:49.182752 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5a374-40db-4b32-b54b-f7e813631b3b None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:39:49.184116 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 747/1503] 10.222.0.26 () {54 vars in 964 bytes} [Wed Aug 17 23:39:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:39:49.189571 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0cc0b-2939-4fbd-aa7a-14e1a43ae88e None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:39:49.189852 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0cc0b-2939-4fbd-aa7a-14e1a43ae88e None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:39:49.190118 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0cc0b-2939-4fbd-aa7a-14e1a43ae88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:39:49.220213 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:39:49.263846 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-a9e0cc0b-2939-4fbd-aa7a-14e1a43ae88e None None] MFA Rules not processed for user `62906d50b6e247c59b69809d3959d4de`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:39:49.265083 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-a9e0cc0b-2939-4fbd-aa7a-14e1a43ae88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:39:49.338130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 757/1504] 10.222.0.26 () {58 vars in 1061 bytes} [Wed Aug 17 23:39:49 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:39:49.346513 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5a374-40db-4b32-b54b-f7e813631b3b None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:39:49.347989 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-e6f5a374-40db-4b32-b54b-f7e813631b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:39:49.401683 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:39:49.458261 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5a374-40db-4b32-b54b-f7e813631b3b None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:39:49.529579 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc6249-afdd-427b-8865-4eefbdb75242 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:39:49.530707 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc6249-afdd-427b-8865-4eefbdb75242 service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:39:49.530981 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc6249-afdd-427b-8865-4eefbdb75242 service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:39:49.531244 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc6249-afdd-427b-8865-4eefbdb75242 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:39:49.533119 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-4fcc6249-afdd-427b-8865-4eefbdb75242 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:39:49.628391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcc6249-afdd-427b-8865-4eefbdb75242 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:39:49.629762 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcc6249-afdd-427b-8865-4eefbdb75242 service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:39:49.633209 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:39:49.647765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 748/1505] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:39:49 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:39:49.693000 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e83ca-c581-4929-8542-50d6191efc7f None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:39:49.693275 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e83ca-c581-4929-8542-50d6191efc7f None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:39:49.693549 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e83ca-c581-4929-8542-50d6191efc7f None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:39:49.694855 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 758/1506] 10.222.0.26 () {52 vars in 830 bytes} [Wed Aug 17 23:39:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:39:49.698425 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc6249-afdd-427b-8865-4eefbdb75242 service cinder] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:39:49.699659 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-4fcc6249-afdd-427b-8865-4eefbdb75242 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:39:49.797980 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc6249-afdd-427b-8865-4eefbdb75242 service cinder] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:39:49.887661 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc583d-ca06-431e-a158-c2904a6084bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:39:49.888246 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc583d-ca06-431e-a158-c2904a6084bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:39:49.888519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc583d-ca06-431e-a158-c2904a6084bf admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:39:49.888781 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc583d-ca06-431e-a158-c2904a6084bf admin admin] PATH_INFO: `/v3/users/4d9d381b253647dd8e312d6ac2121c35` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:39:49.900918 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dc583d-ca06-431e-a158-c2904a6084bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d9d381b253647dd8e312d6ac2121c35)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:39:49.901936 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: WARNING py.warnings [None req-40dc583d-ca06-431e-a158-c2904a6084bf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 17 23:39:49.901936 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: warnings.warn(msg) Aug 17 23:39:49.901936 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: Aug 17 23:39:49.903138 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dc583d-ca06-431e-a158-c2904a6084bf admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:39:49.914150 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:39:49.968878 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-40dc583d-ca06-431e-a158-c2904a6084bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d9d381b253647dd8e312d6ac2121c35'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:39:49.969311 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-40dc583d-ca06-431e-a158-c2904a6084bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d9d381b253647dd8e312d6ac2121c35'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:39:49.991381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-40dc583d-ca06-431e-a158-c2904a6084bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d9d381b253647dd8e312d6ac2121c35'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:39:50.000791 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-40dc583d-ca06-431e-a158-c2904a6084bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d9d381b253647dd8e312d6ac2121c35'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:39:50.001492 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-40dc583d-ca06-431e-a158-c2904a6084bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d9d381b253647dd8e312d6ac2121c35'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:39:50.022282 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 749/1507] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:39:49 2022] DELETE /identity/v3/users/4d9d381b253647dd8e312d6ac2121c35 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:39:50.026627 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af0fcb-5304-4957-bd31-eb4cce3b5b6e None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:39:50.027078 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af0fcb-5304-4957-bd31-eb4cce3b5b6e None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:39:50.027187 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af0fcb-5304-4957-bd31-eb4cce3b5b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:39:50.087584 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-c6af0fcb-5304-4957-bd31-eb4cce3b5b6e None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:39:50.088815 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-c6af0fcb-5304-4957-bd31-eb4cce3b5b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:39:50.162151 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:39:50.175332 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 759/1508] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:39:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:39:50.185508 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc583d-ca06-431e-a158-c2904a6084bf admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:39:50.187260 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-40dc583d-ca06-431e-a158-c2904a6084bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:39:50.285224 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc583d-ca06-431e-a158-c2904a6084bf admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:39:50.338369 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:39:50.360793 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a25910-9cf8-4b1b-81ab-85ea8a165b2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:39:50.361407 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a25910-9cf8-4b1b-81ab-85ea8a165b2e service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:39:50.361687 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a25910-9cf8-4b1b-81ab-85ea8a165b2e service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:39:50.361954 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a25910-9cf8-4b1b-81ab-85ea8a165b2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:39:50.363544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-04a25910-9cf8-4b1b-81ab-85ea8a165b2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:39:50.470556 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a25910-9cf8-4b1b-81ab-85ea8a165b2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:39:50.472034 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a25910-9cf8-4b1b-81ab-85ea8a165b2e service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:39:50.490752 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 750/1509] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:39:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:39:50.846195 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af0fcb-5304-4957-bd31-eb4cce3b5b6e None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:39:50.847549 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-c6af0fcb-5304-4957-bd31-eb4cce3b5b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:39:50.858965 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af0fcb-5304-4957-bd31-eb4cce3b5b6e None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:39:50.954099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb32692e-85cf-49bc-b19d-d6892197d471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:39:50.954731 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32692e-85cf-49bc-b19d-d6892197d471 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:39:50.955034 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32692e-85cf-49bc-b19d-d6892197d471 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:39:50.955302 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32692e-85cf-49bc-b19d-d6892197d471 admin admin] PATH_INFO: `/v3/projects/9892eb3314e645d0a2d78266d38323cb` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:39:50.965219 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb32692e-85cf-49bc-b19d-d6892197d471 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9892eb3314e645d0a2d78266d38323cb)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:39:50.966192 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: WARNING py.warnings [None req-cb32692e-85cf-49bc-b19d-d6892197d471 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 17 23:39:50.966192 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: warnings.warn(msg) Aug 17 23:39:50.966192 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: Aug 17 23:39:50.967401 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb32692e-85cf-49bc-b19d-d6892197d471 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:39:51.025066 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-cb32692e-85cf-49bc-b19d-d6892197d471 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9892eb3314e645d0a2d78266d38323cb', 'request_id': 'req-cb32692e-85cf-49bc-b19d-d6892197d471'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:39:51.031012 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-cb32692e-85cf-49bc-b19d-d6892197d471 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9892eb3314e645d0a2d78266d38323cb', 'request_id': 'req-cb32692e-85cf-49bc-b19d-d6892197d471'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:39:51.041588 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-cb32692e-85cf-49bc-b19d-d6892197d471 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9892eb3314e645d0a2d78266d38323cb', 'request_id': 'req-cb32692e-85cf-49bc-b19d-d6892197d471'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:39:51.071204 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-cb32692e-85cf-49bc-b19d-d6892197d471 admin admin] The token cache is being invalidate because project 9892eb3314e645d0a2d78266d38323cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:39:51.071495 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-cb32692e-85cf-49bc-b19d-d6892197d471 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:39:51.072057 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-cb32692e-85cf-49bc-b19d-d6892197d471 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:39:51.073356 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 760/1510] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:39:50 2022] DELETE /identity/v3/projects/9892eb3314e645d0a2d78266d38323cb => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:40:00.488551 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=62 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:40:01.069221 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:41:17.141532 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e81a1-714d-4e5d-99b4-99613d377935 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:41:17.142127 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e81a1-714d-4e5d-99b4-99613d377935 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:41:17.142127 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e81a1-714d-4e5d-99b4-99613d377935 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:41:17.143400 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 751/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 17 23:41:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:41:17.148161 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4266e85-fd47-4b41-b89c-06631d8190bc None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:41:17.148618 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4266e85-fd47-4b41-b89c-06631d8190bc None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:41:17.148726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4266e85-fd47-4b41-b89c-06631d8190bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:41:17.228494 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-c4266e85-fd47-4b41-b89c-06631d8190bc None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:41:17.229688 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-c4266e85-fd47-4b41-b89c-06631d8190bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:41:17.329651 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 761/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 17 23:41:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:41:17.482876 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608e81a1-714d-4e5d-99b4-99613d377935 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:41:17.485037 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-608e81a1-714d-4e5d-99b4-99613d377935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:41:17.600933 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608e81a1-714d-4e5d-99b4-99613d377935 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:41:17.606318 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0664309-3625-40a8-a109-f6cf0eb927a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:41:17.606920 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0664309-3625-40a8-a109-f6cf0eb927a4 service nova] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:41:17.607189 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0664309-3625-40a8-a109-f6cf0eb927a4 service nova] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:41:17.607455 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0664309-3625-40a8-a109-f6cf0eb927a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:41:17.615296 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0664309-3625-40a8-a109-f6cf0eb927a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:41:17.616903 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0664309-3625-40a8-a109-f6cf0eb927a4 service nova] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:41:17.626035 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 752/1513] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:41:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:41:18.450331 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed354b01-f82e-40e6-aa0c-2eb1a1cbac31 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:41:18.450932 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed354b01-f82e-40e6-aa0c-2eb1a1cbac31 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:41:18.450932 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed354b01-f82e-40e6-aa0c-2eb1a1cbac31 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:41:18.452332 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 762/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 17 23:41:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:41:18.457120 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d43fc-4b27-4dab-824f-c160ca7af3f8 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:41:18.457560 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d43fc-4b27-4dab-824f-c160ca7af3f8 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:41:18.457752 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d43fc-4b27-4dab-824f-c160ca7af3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:41:18.476437 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-f68d43fc-4b27-4dab-824f-c160ca7af3f8 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:41:18.477630 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-f68d43fc-4b27-4dab-824f-c160ca7af3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:41:18.485929 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 753/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 17 23:41:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:41:18.635830 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed354b01-f82e-40e6-aa0c-2eb1a1cbac31 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:41:18.650032 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed354b01-f82e-40e6-aa0c-2eb1a1cbac31 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:41:18.655909 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20e824-611c-4140-bd67-2ece49deb509 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:41:18.656530 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20e824-611c-4140-bd67-2ece49deb509 service nova] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:41:18.656806 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20e824-611c-4140-bd67-2ece49deb509 service nova] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:41:18.657073 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20e824-611c-4140-bd67-2ece49deb509 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:41:18.665385 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef20e824-611c-4140-bd67-2ece49deb509 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:41:18.666990 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef20e824-611c-4140-bd67-2ece49deb509 service nova] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:41:18.676888 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 763/1516] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:41:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:41:18.984201 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf2520-2bd2-4521-ba45-1274f6758f88 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:41:18.984489 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf2520-2bd2-4521-ba45-1274f6758f88 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:41:18.984764 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf2520-2bd2-4521-ba45-1274f6758f88 None None] PATH_INFO: `/v3` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:41:18.986050 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 754/1517] 10.222.0.18 () {54 vars in 925 bytes} [Wed Aug 17 23:41:18 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:41:18.993003 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6d0d2-b0e1-4ff5-9886-85f55f58abcb None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:41:18.993276 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6d0d2-b0e1-4ff5-9886-85f55f58abcb None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:41:18.993546 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6d0d2-b0e1-4ff5-9886-85f55f58abcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:41:19.051073 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-acf6d0d2-b0e1-4ff5-9886-85f55f58abcb None None] MFA Rules not processed for user `1fc57d911a56423e96c736bc96270ba8`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:41:19.052299 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-acf6d0d2-b0e1-4ff5-9886-85f55f58abcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:41:19.068493 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:41:19.122532 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 764/1518] 10.222.0.18 () {58 vars in 1015 bytes} [Wed Aug 17 23:41:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:41:28.290985 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc074d0c-da96-4765-875d-a9f04f597efa None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:41:28.291541 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc074d0c-da96-4765-875d-a9f04f597efa None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:41:28.291541 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc074d0c-da96-4765-875d-a9f04f597efa None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:41:28.292806 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 755/1519] 10.222.0.26 () {54 vars in 961 bytes} [Wed Aug 17 23:41:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:41:28.297984 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5cc9d-0cee-4a2f-9308-f59248b1db74 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:41:28.298458 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5cc9d-0cee-4a2f-9308-f59248b1db74 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:41:28.298562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5cc9d-0cee-4a2f-9308-f59248b1db74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:41:28.319772 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-dae5cc9d-0cee-4a2f-9308-f59248b1db74 None None] MFA Rules not processed for user `1fc57d911a56423e96c736bc96270ba8`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:41:28.321060 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-dae5cc9d-0cee-4a2f-9308-f59248b1db74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:41:28.330079 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 765/1520] 10.222.0.26 () {58 vars in 1058 bytes} [Wed Aug 17 23:41:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:41:28.337499 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc074d0c-da96-4765-875d-a9f04f597efa None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:41:28.350645 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc074d0c-da96-4765-875d-a9f04f597efa None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:41:28.355476 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967b8ef6-a3ba-464a-9117-aa152b903bf9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc57d911a56423e96c736bc96270ba8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:41:28.356087 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b8ef6-a3ba-464a-9117-aa152b903bf9 service placement] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:41:28.356366 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b8ef6-a3ba-464a-9117-aa152b903bf9 service placement] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:41:28.356635 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b8ef6-a3ba-464a-9117-aa152b903bf9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:41:28.364647 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967b8ef6-a3ba-464a-9117-aa152b903bf9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:41:28.366247 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967b8ef6-a3ba-464a-9117-aa152b903bf9 service placement] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:41:28.376192 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 756/1521] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:41:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:41:29.884364 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba62e6-ad6a-49a8-b422-30b24612d4b6 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:41:29.884946 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba62e6-ad6a-49a8-b422-30b24612d4b6 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:41:29.884946 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba62e6-ad6a-49a8-b422-30b24612d4b6 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:41:29.886265 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 766/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 17 23:41:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:41:29.891547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269efa7-d7a5-4892-8419-35f0d3ad8640 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:41:29.891992 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269efa7-d7a5-4892-8419-35f0d3ad8640 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:41:29.892101 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269efa7-d7a5-4892-8419-35f0d3ad8640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:41:29.912475 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-d269efa7-d7a5-4892-8419-35f0d3ad8640 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:41:29.913659 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-d269efa7-d7a5-4892-8419-35f0d3ad8640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:41:29.921591 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 757/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 17 23:41:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:41:30.076221 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ba62e6-ad6a-49a8-b422-30b24612d4b6 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:41:30.090326 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ba62e6-ad6a-49a8-b422-30b24612d4b6 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:41:30.095643 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac18f8-d063-4080-92df-aef718cfa4cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:41:30.096271 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac18f8-d063-4080-92df-aef718cfa4cc service nova] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:41:30.096549 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac18f8-d063-4080-92df-aef718cfa4cc service nova] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:41:30.096820 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac18f8-d063-4080-92df-aef718cfa4cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:41:30.104757 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ac18f8-d063-4080-92df-aef718cfa4cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:41:30.106370 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ac18f8-d063-4080-92df-aef718cfa4cc service nova] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:41:30.115560 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 767/1524] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:41:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:41:30.940558 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac0159-f93e-4958-989e-f3091bd4882f None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:41:30.941132 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac0159-f93e-4958-989e-f3091bd4882f None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:41:30.941132 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac0159-f93e-4958-989e-f3091bd4882f None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:41:30.942451 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 758/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 17 23:41:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:41:30.947211 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1be83-093e-4d2a-962d-7df56fabc037 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:41:30.947666 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1be83-093e-4d2a-962d-7df56fabc037 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:41:30.947781 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1be83-093e-4d2a-962d-7df56fabc037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:41:30.967686 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-77c1be83-093e-4d2a-962d-7df56fabc037 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:41:30.968982 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-77c1be83-093e-4d2a-962d-7df56fabc037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:41:30.976956 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 768/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 17 23:41:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:41:31.127681 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac0159-f93e-4958-989e-f3091bd4882f None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:41:31.142416 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac0159-f93e-4958-989e-f3091bd4882f None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:41:31.147759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb5b42f-7704-4298-80cc-143fedde94ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:41:31.148370 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5b42f-7704-4298-80cc-143fedde94ea service nova] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:41:31.148646 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5b42f-7704-4298-80cc-143fedde94ea service nova] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:41:31.148916 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5b42f-7704-4298-80cc-143fedde94ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:41:31.157713 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb5b42f-7704-4298-80cc-143fedde94ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:41:31.159711 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb5b42f-7704-4298-80cc-143fedde94ea service nova] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:41:31.168879 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 759/1527] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:41:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:41:34.852421 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ec73e-5b15-4b81-8f14-7333aedfd1b2 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:41:34.852995 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ec73e-5b15-4b81-8f14-7333aedfd1b2 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:41:34.852995 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ec73e-5b15-4b81-8f14-7333aedfd1b2 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:41:34.854585 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 769/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 23:41:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:41:34.859581 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75fab12-b166-4ff6-9948-9f6ae0e19285 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:41:34.860008 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75fab12-b166-4ff6-9948-9f6ae0e19285 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:41:34.860190 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75fab12-b166-4ff6-9948-9f6ae0e19285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:41:34.881551 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-c75fab12-b166-4ff6-9948-9f6ae0e19285 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:41:34.882984 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-c75fab12-b166-4ff6-9948-9f6ae0e19285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:41:34.891379 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 760/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 23:41:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:41:35.179565 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ec73e-5b15-4b81-8f14-7333aedfd1b2 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:41:35.181362 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-151ec73e-5b15-4b81-8f14-7333aedfd1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:41:35.258221 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ec73e-5b15-4b81-8f14-7333aedfd1b2 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:41:35.261711 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee167910-97b6-4d87-9e72-7bef25c6606c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:41:35.262313 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee167910-97b6-4d87-9e72-7bef25c6606c service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:41:35.262595 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee167910-97b6-4d87-9e72-7bef25c6606c service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:41:35.262871 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee167910-97b6-4d87-9e72-7bef25c6606c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:41:35.272299 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee167910-97b6-4d87-9e72-7bef25c6606c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:41:35.273734 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee167910-97b6-4d87-9e72-7bef25c6606c service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:41:35.282952 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 770/1530] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:41:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:41:37.156238 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4690121-81e7-4cdf-8665-c402de6c8528 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:41:37.156780 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4690121-81e7-4cdf-8665-c402de6c8528 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:41:37.156780 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4690121-81e7-4cdf-8665-c402de6c8528 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:41:37.158057 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 761/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 23:41:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:41:37.163460 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e78e2-a32d-4f19-b0ea-8bfb159483b9 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:41:37.163897 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e78e2-a32d-4f19-b0ea-8bfb159483b9 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:41:37.164049 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e78e2-a32d-4f19-b0ea-8bfb159483b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:41:37.183352 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-613e78e2-a32d-4f19-b0ea-8bfb159483b9 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:41:37.184608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-613e78e2-a32d-4f19-b0ea-8bfb159483b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:41:37.193354 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 771/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 23:41:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:41:37.478602 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4690121-81e7-4cdf-8665-c402de6c8528 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:41:37.489836 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4690121-81e7-4cdf-8665-c402de6c8528 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:41:37.493288 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead32d94-cbf8-4524-8847-07eb7c5cafc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:41:37.493867 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead32d94-cbf8-4524-8847-07eb7c5cafc6 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:41:37.494139 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead32d94-cbf8-4524-8847-07eb7c5cafc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:41:37.494403 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead32d94-cbf8-4524-8847-07eb7c5cafc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:41:37.503015 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead32d94-cbf8-4524-8847-07eb7c5cafc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:41:37.504467 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead32d94-cbf8-4524-8847-07eb7c5cafc6 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:41:37.513621 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 762/1533] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:41:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:41:40.990968 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61f015-c037-452a-8db5-923c4b76cc72 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:41:40.991527 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61f015-c037-452a-8db5-923c4b76cc72 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:41:40.991527 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61f015-c037-452a-8db5-923c4b76cc72 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:41:40.992784 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 772/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 23:41:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:41:40.998322 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fb8a6-a4db-47c0-91dd-389ad3e85cd6 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:41:40.998719 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fb8a6-a4db-47c0-91dd-389ad3e85cd6 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:41:40.998851 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fb8a6-a4db-47c0-91dd-389ad3e85cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:41:41.006347 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=81 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:41:41.018677 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-774fb8a6-a4db-47c0-91dd-389ad3e85cd6 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:41:41.019872 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-774fb8a6-a4db-47c0-91dd-389ad3e85cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:41:41.027709 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 763/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 23:41:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:41:41.572354 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61f015-c037-452a-8db5-923c4b76cc72 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:41:41.574016 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-ec61f015-c037-452a-8db5-923c4b76cc72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:41:41.636981 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:41:41.650289 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61f015-c037-452a-8db5-923c4b76cc72 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:41:41.653814 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a51b7-1686-446e-bff4-babcf292dc03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:41:41.654399 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a51b7-1686-446e-bff4-babcf292dc03 service cinder] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:41:41.654671 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a51b7-1686-446e-bff4-babcf292dc03 service cinder] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:41:41.654939 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a51b7-1686-446e-bff4-babcf292dc03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:41:41.663812 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a51b7-1686-446e-bff4-babcf292dc03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:41:41.665265 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a51b7-1686-446e-bff4-babcf292dc03 service cinder] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:41:41.674942 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 773/1536] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:41:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:41:43.583503 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dcbb86-f968-4f01-807a-fe9650beadbc None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:41:43.584082 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dcbb86-f968-4f01-807a-fe9650beadbc None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:41:43.584082 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dcbb86-f968-4f01-807a-fe9650beadbc None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:41:43.585418 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 764/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 23:41:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:41:43.590609 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97797b-d487-41d0-9d1b-e93856ef4118 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:41:43.591006 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97797b-d487-41d0-9d1b-e93856ef4118 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:41:43.591158 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97797b-d487-41d0-9d1b-e93856ef4118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:41:43.610713 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-de97797b-d487-41d0-9d1b-e93856ef4118 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:41:43.611905 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-de97797b-d487-41d0-9d1b-e93856ef4118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:41:43.619480 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 774/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 23:41:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:41:44.170720 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dcbb86-f968-4f01-807a-fe9650beadbc None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:41:44.172394 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-c4dcbb86-f968-4f01-807a-fe9650beadbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:41:44.186829 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dcbb86-f968-4f01-807a-fe9650beadbc None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:41:44.190785 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdad8cd-b296-4b6c-8858-a2c366e1fba1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:41:44.191421 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdad8cd-b296-4b6c-8858-a2c366e1fba1 service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:41:44.191694 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdad8cd-b296-4b6c-8858-a2c366e1fba1 service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:41:44.191984 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdad8cd-b296-4b6c-8858-a2c366e1fba1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:41:44.202528 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdad8cd-b296-4b6c-8858-a2c366e1fba1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:41:44.204047 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdad8cd-b296-4b6c-8858-a2c366e1fba1 service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:41:44.213455 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 765/1539] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:41:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:41:53.605947 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=11 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:41:54.209399 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=8 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:07.333360 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e300b7f-ea4d-47ed-924e-95fe78258a55 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:07.334490 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e300b7f-ea4d-47ed-924e-95fe78258a55 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:07.335024 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e300b7f-ea4d-47ed-924e-95fe78258a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:07.361045 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-0e300b7f-ea4d-47ed-924e-95fe78258a55 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:07.362278 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-0e300b7f-ea4d-47ed-924e-95fe78258a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:07.375126 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 775/1540] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:07.384158 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7ef7d-cede-4677-ad09-7f2355d678ee None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:07.384720 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7ef7d-cede-4677-ad09-7f2355d678ee None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:07.384820 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7ef7d-cede-4677-ad09-7f2355d678ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:07.425623 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-93d7ef7d-cede-4677-ad09-7f2355d678ee None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:07.426920 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-93d7ef7d-cede-4677-ad09-7f2355d678ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:07.440777 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 766/1541] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:07.457571 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e300b7f-ea4d-47ed-924e-95fe78258a55 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:07.472612 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e300b7f-ea4d-47ed-924e-95fe78258a55 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:07.480608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95addbf2-336d-4411-9ad5-874638d63066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:07.481258 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95addbf2-336d-4411-9ad5-874638d63066 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:07.481535 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95addbf2-336d-4411-9ad5-874638d63066 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:07.481807 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95addbf2-336d-4411-9ad5-874638d63066 admin admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:07.482648 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95addbf2-336d-4411-9ad5-874638d63066 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:43:07.482966 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95addbf2-336d-4411-9ad5-874638d63066 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:07.484683 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95addbf2-336d-4411-9ad5-874638d63066 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:07.509283 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 776/1542] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:43:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:43:07.516568 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832671ef-e11d-49f1-9f6e-e2c4a6841c8b None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:07.516913 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832671ef-e11d-49f1-9f6e-e2c4a6841c8b None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:07.517114 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832671ef-e11d-49f1-9f6e-e2c4a6841c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:07.520914 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95addbf2-336d-4411-9ad5-874638d63066 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:07.538083 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95addbf2-336d-4411-9ad5-874638d63066 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:07.546924 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8ce12-c2c5-4c1d-bf9b-79dc07ecdc9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:07.547583 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8ce12-c2c5-4c1d-bf9b-79dc07ecdc9d admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:07.547849 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8ce12-c2c5-4c1d-bf9b-79dc07ecdc9d admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:07.548161 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8ce12-c2c5-4c1d-bf9b-79dc07ecdc9d admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:07.548791 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b8ce12-c2c5-4c1d-bf9b-79dc07ecdc9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:07.549900 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: WARNING py.warnings [None req-b2b8ce12-c2c5-4c1d-bf9b-79dc07ecdc9d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 17 23:43:07.549900 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: warnings.warn(msg) Aug 17 23:43:07.549900 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: Aug 17 23:43:07.551042 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b8ce12-c2c5-4c1d-bf9b-79dc07ecdc9d admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:07.558024 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-832671ef-e11d-49f1-9f6e-e2c4a6841c8b None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:07.559279 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-832671ef-e11d-49f1-9f6e-e2c4a6841c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:07.578832 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 767/1543] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:07.580170 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408c7ad-5551-488f-9cd0-f57222ca85b2 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:07.580501 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408c7ad-5551-488f-9cd0-f57222ca85b2 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:07.580723 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408c7ad-5551-488f-9cd0-f57222ca85b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:07.618868 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-9408c7ad-5551-488f-9cd0-f57222ca85b2 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:07.620852 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-9408c7ad-5551-488f-9cd0-f57222ca85b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:07.621934 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 777/1544] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:43:07 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:07.623372 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fdd0c8-a67b-4430-bf1d-4e41b3f9e72a None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:07.623659 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fdd0c8-a67b-4430-bf1d-4e41b3f9e72a None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:07.623935 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fdd0c8-a67b-4430-bf1d-4e41b3f9e72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:07.651292 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-88fdd0c8-a67b-4430-bf1d-4e41b3f9e72a None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:07.652708 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-88fdd0c8-a67b-4430-bf1d-4e41b3f9e72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:07.739559 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 768/1545] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:07.740244 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9408c7ad-5551-488f-9cd0-f57222ca85b2 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:07.754155 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9408c7ad-5551-488f-9cd0-f57222ca85b2 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:07.758090 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 778/1546] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:07.759365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d389c8-5c62-41c7-b703-415b01a7a265 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:07.759660 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d389c8-5c62-41c7-b703-415b01a7a265 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:07.759932 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d389c8-5c62-41c7-b703-415b01a7a265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:07.768673 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b4791-8b6d-4a73-b623-1548659f7cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:07.769312 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b4791-8b6d-4a73-b623-1548659f7cf5 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:07.769593 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b4791-8b6d-4a73-b623-1548659f7cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:07.769871 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b4791-8b6d-4a73-b623-1548659f7cf5 admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:07.770511 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86b4791-8b6d-4a73-b623-1548659f7cf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:07.771889 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: WARNING py.warnings [None req-c86b4791-8b6d-4a73-b623-1548659f7cf5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 17 23:43:07.771889 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: warnings.warn(msg) Aug 17 23:43:07.771889 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: Aug 17 23:43:07.773152 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86b4791-8b6d-4a73-b623-1548659f7cf5 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:07.806431 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-f0d389c8-5c62-41c7-b703-415b01a7a265 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:07.808354 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-f0d389c8-5c62-41c7-b703-415b01a7a265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:07.815743 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 769/1547] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:43:07 2022] POST /identity/v3/users => generated 293 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:07.819935 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60909877-b35b-4a19-9b78-54a2c879c2da None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:07.820242 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60909877-b35b-4a19-9b78-54a2c879c2da None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:07.820515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60909877-b35b-4a19-9b78-54a2c879c2da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:07.826275 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 779/1548] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:07.826275 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d389c8-5c62-41c7-b703-415b01a7a265 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:07.843265 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d389c8-5c62-41c7-b703-415b01a7a265 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:07.848579 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf37174-597a-471f-b751-f77809eb98ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:07.849191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf37174-597a-471f-b751-f77809eb98ce admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:07.849471 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf37174-597a-471f-b751-f77809eb98ce admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:07.849744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf37174-597a-471f-b751-f77809eb98ce admin admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:07.850395 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf37174-597a-471f-b751-f77809eb98ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:43:07.850700 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf37174-597a-471f-b751-f77809eb98ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:07.851607 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-60909877-b35b-4a19-9b78-54a2c879c2da None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:07.852334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf37174-597a-471f-b751-f77809eb98ce admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:07.853138 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-60909877-b35b-4a19-9b78-54a2c879c2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:07.863041 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 770/1549] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:07.866757 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1590075-5ac6-49f7-8eb1-fb6b2ebd6b78 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:07.866757 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1590075-5ac6-49f7-8eb1-fb6b2ebd6b78 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:07.866757 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1590075-5ac6-49f7-8eb1-fb6b2ebd6b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:07.869378 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 780/1550] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:43:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:43:07.871532 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097dd29e-7ece-49e0-af65-8c69aefed80e None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:07.871998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097dd29e-7ece-49e0-af65-8c69aefed80e None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:07.872428 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097dd29e-7ece-49e0-af65-8c69aefed80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:07.885831 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-f1590075-5ac6-49f7-8eb1-fb6b2ebd6b78 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:07.887022 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-f1590075-5ac6-49f7-8eb1-fb6b2ebd6b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:07.891527 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-097dd29e-7ece-49e0-af65-8c69aefed80e None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:07.892747 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-097dd29e-7ece-49e0-af65-8c69aefed80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:07.894826 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 771/1551] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:07.895591 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1590075-5ac6-49f7-8eb1-fb6b2ebd6b78 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:07.904927 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1590075-5ac6-49f7-8eb1-fb6b2ebd6b78 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:07.906510 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 781/1552] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:07.907687 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6139420-4949-4049-9f6b-b2ed017b69bf None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:07.908002 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6139420-4949-4049-9f6b-b2ed017b69bf None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:07.908300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6139420-4949-4049-9f6b-b2ed017b69bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:07.915590 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a501df-5501-49b5-a575-65efc6532af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:07.916226 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a501df-5501-49b5-a575-65efc6532af3 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:07.917048 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a501df-5501-49b5-a575-65efc6532af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:07.917308 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a501df-5501-49b5-a575-65efc6532af3 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:07.917986 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a501df-5501-49b5-a575-65efc6532af3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:07.919612 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a501df-5501-49b5-a575-65efc6532af3 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:07.929447 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-b6139420-4949-4049-9f6b-b2ed017b69bf None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:07.933443 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 772/1553] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:07.933599 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-b6139420-4949-4049-9f6b-b2ed017b69bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:07.933701 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9918b46-9118-471c-acf5-c01ab5b6b1b3 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:07.933701 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9918b46-9118-471c-acf5-c01ab5b6b1b3 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:07.933701 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9918b46-9118-471c-acf5-c01ab5b6b1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:07.939396 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 782/1554] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:07.940215 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6139420-4949-4049-9f6b-b2ed017b69bf None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:07.951308 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-a9918b46-9118-471c-acf5-c01ab5b6b1b3 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:07.951488 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6139420-4949-4049-9f6b-b2ed017b69bf None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:07.952524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-a9918b46-9118-471c-acf5-c01ab5b6b1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:07.955933 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d2e28-04f2-4c1e-b046-10073707165a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:07.956545 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d2e28-04f2-4c1e-b046-10073707165a admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:07.956820 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d2e28-04f2-4c1e-b046-10073707165a admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:07.957113 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d2e28-04f2-4c1e-b046-10073707165a admin admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:07.957748 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d2e28-04f2-4c1e-b046-10073707165a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:43:07.958043 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d2e28-04f2-4c1e-b046-10073707165a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:07.959603 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d2e28-04f2-4c1e-b046-10073707165a admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:07.964560 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 773/1555] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:07.964560 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf978a7-d156-48aa-9f19-1d20b6567743 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:07.964560 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf978a7-d156-48aa-9f19-1d20b6567743 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:07.964560 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf978a7-d156-48aa-9f19-1d20b6567743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:07.978093 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 783/1556] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:43:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:43:07.978093 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d2e28-04f2-4c1e-b046-10073707165a admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:07.982071 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d2e28-04f2-4c1e-b046-10073707165a admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:07.983954 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-1cf978a7-d156-48aa-9f19-1d20b6567743 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:07.985162 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-1cf978a7-d156-48aa-9f19-1d20b6567743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:07.986737 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c2bbb0-0189-4cc3-afdc-a7453cfc0d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:07.987329 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2bbb0-0189-4cc3-afdc-a7453cfc0d84 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:07.987608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2bbb0-0189-4cc3-afdc-a7453cfc0d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:07.987880 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2bbb0-0189-4cc3-afdc-a7453cfc0d84 admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:07.988532 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c2bbb0-0189-4cc3-afdc-a7453cfc0d84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:07.990471 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c2bbb0-0189-4cc3-afdc-a7453cfc0d84 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:07.992953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 774/1557] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:07.994102 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cce9ff-80a7-4df8-85aa-c796f2263357 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:07.994381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cce9ff-80a7-4df8-85aa-c796f2263357 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:07.994677 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cce9ff-80a7-4df8-85aa-c796f2263357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:08.013977 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-e8cce9ff-80a7-4df8-85aa-c796f2263357 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:08.015180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-e8cce9ff-80a7-4df8-85aa-c796f2263357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:08.019749 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 784/1558] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:43:07 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:08.021020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea202fa9-2047-4682-832c-babcaf3c0dcd None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:08.021301 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea202fa9-2047-4682-832c-babcaf3c0dcd None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:08.021583 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea202fa9-2047-4682-832c-babcaf3c0dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:08.023318 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 775/1559] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:08.024024 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cce9ff-80a7-4df8-85aa-c796f2263357 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:08.033973 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cce9ff-80a7-4df8-85aa-c796f2263357 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:08.040965 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-ea202fa9-2047-4682-832c-babcaf3c0dcd None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:08.042723 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-ea202fa9-2047-4682-832c-babcaf3c0dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:08.084440 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:08.095557 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de463a37-e117-484e-a3cd-3c0547cc7c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:08.096190 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de463a37-e117-484e-a3cd-3c0547cc7c6a admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:08.096464 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de463a37-e117-484e-a3cd-3c0547cc7c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:08.096733 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de463a37-e117-484e-a3cd-3c0547cc7c6a admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:08.097369 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de463a37-e117-484e-a3cd-3c0547cc7c6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:08.098956 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de463a37-e117-484e-a3cd-3c0547cc7c6a admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:08.109118 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 776/1560] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:08.109748 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 785/1561] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:08.110435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea202fa9-2047-4682-832c-babcaf3c0dcd None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:08.113439 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8840cdb-414e-4562-9111-86184294fa01 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:08.113725 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8840cdb-414e-4562-9111-86184294fa01 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:08.113999 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8840cdb-414e-4562-9111-86184294fa01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:08.120164 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea202fa9-2047-4682-832c-babcaf3c0dcd None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:08.124654 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374cc7a0-5019-4117-bfdb-8155407f7bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:08.125242 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374cc7a0-5019-4117-bfdb-8155407f7bfc admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:08.125542 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374cc7a0-5019-4117-bfdb-8155407f7bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:08.125816 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374cc7a0-5019-4117-bfdb-8155407f7bfc admin admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:08.126456 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374cc7a0-5019-4117-bfdb-8155407f7bfc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:43:08.126757 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374cc7a0-5019-4117-bfdb-8155407f7bfc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:08.128420 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374cc7a0-5019-4117-bfdb-8155407f7bfc admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:08.132767 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-e8840cdb-414e-4562-9111-86184294fa01 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:08.133964 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-e8840cdb-414e-4562-9111-86184294fa01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:08.139937 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 786/1562] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:43:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:43:08.142244 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 777/1563] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:08.144856 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374cc7a0-5019-4117-bfdb-8155407f7bfc admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:08.145848 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a90f8c2-dee6-4367-90fd-9a5a297ab248 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:08.146140 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a90f8c2-dee6-4367-90fd-9a5a297ab248 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:08.146413 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a90f8c2-dee6-4367-90fd-9a5a297ab248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:08.155524 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374cc7a0-5019-4117-bfdb-8155407f7bfc admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:08.160326 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc36e57-e22c-471d-887c-6ea23dd17f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:08.160914 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc36e57-e22c-471d-887c-6ea23dd17f6c admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:08.161191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc36e57-e22c-471d-887c-6ea23dd17f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:08.161464 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc36e57-e22c-471d-887c-6ea23dd17f6c admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:08.162082 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc36e57-e22c-471d-887c-6ea23dd17f6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:08.164072 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc36e57-e22c-471d-887c-6ea23dd17f6c admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:08.165859 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-3a90f8c2-dee6-4367-90fd-9a5a297ab248 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:08.167571 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-3a90f8c2-dee6-4367-90fd-9a5a297ab248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:08.175280 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 778/1564] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:08.176042 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a90f8c2-dee6-4367-90fd-9a5a297ab248 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:08.185549 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a90f8c2-dee6-4367-90fd-9a5a297ab248 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:08.190114 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e953d09-8978-4546-85f5-cccba1b7cd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:08.190698 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e953d09-8978-4546-85f5-cccba1b7cd8d admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:08.191109 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e953d09-8978-4546-85f5-cccba1b7cd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:08.191390 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e953d09-8978-4546-85f5-cccba1b7cd8d admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:08.192030 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e953d09-8978-4546-85f5-cccba1b7cd8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:08.192686 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 787/1565] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:43:08 2022] POST /identity/v3/projects => generated 393 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:08.193532 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc36e57-e22c-471d-887c-6ea23dd17f6c admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:08.194015 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e953d09-8978-4546-85f5-cccba1b7cd8d admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:08.203523 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:08.217856 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 779/1566] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:43:08 2022] POST /identity/v3/users => generated 293 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:08.218706 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e953d09-8978-4546-85f5-cccba1b7cd8d admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:08.233644 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e953d09-8978-4546-85f5-cccba1b7cd8d admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:08.257323 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc36e57-e22c-471d-887c-6ea23dd17f6c admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:08.261890 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f4c12a-f175-494a-847a-08cffc4bc8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:08.262488 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f4c12a-f175-494a-847a-08cffc4bc8e6 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:08.262766 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f4c12a-f175-494a-847a-08cffc4bc8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:08.263033 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f4c12a-f175-494a-847a-08cffc4bc8e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:08.263673 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f4c12a-f175-494a-847a-08cffc4bc8e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:43:08.263986 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f4c12a-f175-494a-847a-08cffc4bc8e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:08.265589 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f4c12a-f175-494a-847a-08cffc4bc8e6 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:08.277313 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 788/1567] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:43:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:43:08.278099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f4c12a-f175-494a-847a-08cffc4bc8e6 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:08.287996 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f4c12a-f175-494a-847a-08cffc4bc8e6 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:08.293187 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe4fde4-e7c1-4644-9f83-1314d2929b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:08.293798 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4fde4-e7c1-4644-9f83-1314d2929b3f admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:08.294079 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4fde4-e7c1-4644-9f83-1314d2929b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:08.294353 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4fde4-e7c1-4644-9f83-1314d2929b3f admin admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:08.295008 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe4fde4-e7c1-4644-9f83-1314d2929b3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:43:08.295310 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe4fde4-e7c1-4644-9f83-1314d2929b3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:08.296913 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe4fde4-e7c1-4644-9f83-1314d2929b3f admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:08.298953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4bd51-2517-413a-b25f-a4d9b0b7969f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:08.299945 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4bd51-2517-413a-b25f-a4d9b0b7969f admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:08.300250 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4bd51-2517-413a-b25f-a4d9b0b7969f admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:08.300519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4bd51-2517-413a-b25f-a4d9b0b7969f admin admin] PATH_INFO: `/v3/projects/aaf39b11fdbc4995aecfa322126450fe/users/140b34e8949846549815df130d6ab172/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:08.309049 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 789/1568] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:43:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:43:08.309919 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe4fde4-e7c1-4644-9f83-1314d2929b3f admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:08.314554 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c4bd51-2517-413a-b25f-a4d9b0b7969f admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaf39b11fdbc4995aecfa322126450fe, user_id=140b34e8949846549815df130d6ab172, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:08.317341 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c4bd51-2517-413a-b25f-a4d9b0b7969f admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:08.320032 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe4fde4-e7c1-4644-9f83-1314d2929b3f admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:08.326147 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8896e9f6-0f14-438e-a558-fa142ae405b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:08.326739 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8896e9f6-0f14-438e-a558-fa142ae405b0 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:08.327022 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8896e9f6-0f14-438e-a558-fa142ae405b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:08.327295 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8896e9f6-0f14-438e-a558-fa142ae405b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:08.327928 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8896e9f6-0f14-438e-a558-fa142ae405b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:43:08.328259 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8896e9f6-0f14-438e-a558-fa142ae405b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:08.329837 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8896e9f6-0f14-438e-a558-fa142ae405b0 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:08.330548 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 780/1569] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:43:08 2022] PUT /identity/v3/projects/aaf39b11fdbc4995aecfa322126450fe/users/140b34e8949846549815df130d6ab172/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:08.331323 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4bd51-2517-413a-b25f-a4d9b0b7969f admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:08.350949 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 790/1570] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:43:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:43:08.351777 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8896e9f6-0f14-438e-a558-fa142ae405b0 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:08.361829 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8896e9f6-0f14-438e-a558-fa142ae405b0 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:08.393963 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4bd51-2517-413a-b25f-a4d9b0b7969f admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:08.398481 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e6df16-c7cc-438e-9e36-82ab6d9c316e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:08.399080 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6df16-c7cc-438e-9e36-82ab6d9c316e admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:08.399355 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6df16-c7cc-438e-9e36-82ab6d9c316e admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:08.399623 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6df16-c7cc-438e-9e36-82ab6d9c316e admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:08.400542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e6df16-c7cc-438e-9e36-82ab6d9c316e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:08.402726 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e6df16-c7cc-438e-9e36-82ab6d9c316e admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:08.419450 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07850122-a491-4140-a66f-71afe818b39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:08.420416 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07850122-a491-4140-a66f-71afe818b39a admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:08.420854 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07850122-a491-4140-a66f-71afe818b39a admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:08.421290 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07850122-a491-4140-a66f-71afe818b39a admin admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:08.422082 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07850122-a491-4140-a66f-71afe818b39a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:43:08.422505 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07850122-a491-4140-a66f-71afe818b39a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:08.424322 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07850122-a491-4140-a66f-71afe818b39a admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:08.429286 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 781/1571] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:43:08 2022] POST /identity/v3/projects => generated 389 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:08.430104 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e6df16-c7cc-438e-9e36-82ab6d9c316e admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:08.439718 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 791/1572] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:43:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:43:08.440463 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07850122-a491-4140-a66f-71afe818b39a admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:08.452177 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07850122-a491-4140-a66f-71afe818b39a admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:08.456903 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:08.500169 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e6df16-c7cc-438e-9e36-82ab6d9c316e admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:08.505487 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72745ad-93cf-48a9-9a78-98824b1f1081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:08.506098 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72745ad-93cf-48a9-9a78-98824b1f1081 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:08.506373 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72745ad-93cf-48a9-9a78-98824b1f1081 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:08.506639 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72745ad-93cf-48a9-9a78-98824b1f1081 admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:08.507256 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72745ad-93cf-48a9-9a78-98824b1f1081 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:08.509208 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72745ad-93cf-48a9-9a78-98824b1f1081 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:08.509771 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9289f-900c-4b0c-aae3-74eb71593be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:08.510897 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9289f-900c-4b0c-aae3-74eb71593be2 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:08.511174 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9289f-900c-4b0c-aae3-74eb71593be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:08.511439 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9289f-900c-4b0c-aae3-74eb71593be2 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:08.512109 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af9289f-900c-4b0c-aae3-74eb71593be2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:08.513678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af9289f-900c-4b0c-aae3-74eb71593be2 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:08.517480 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:08.524426 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 792/1573] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:08.528674 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9289f-900c-4b0c-aae3-74eb71593be2 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:08.533152 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 782/1574] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:43:08 2022] POST /identity/v3/users => generated 297 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:08.533999 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72745ad-93cf-48a9-9a78-98824b1f1081 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:08.538470 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9289f-900c-4b0c-aae3-74eb71593be2 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:08.543295 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260fab40-3044-4d88-8cb1-6117c01c3903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:08.543891 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260fab40-3044-4d88-8cb1-6117c01c3903 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:08.544203 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260fab40-3044-4d88-8cb1-6117c01c3903 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:08.544480 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260fab40-3044-4d88-8cb1-6117c01c3903 admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:08.544588 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72745ad-93cf-48a9-9a78-98824b1f1081 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:08.545128 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260fab40-3044-4d88-8cb1-6117c01c3903 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:08.547091 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260fab40-3044-4d88-8cb1-6117c01c3903 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:08.549056 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a810b-cd61-4875-a909-2e77e8ac399e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:08.549669 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a810b-cd61-4875-a909-2e77e8ac399e admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:08.549944 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a810b-cd61-4875-a909-2e77e8ac399e admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:08.550215 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a810b-cd61-4875-a909-2e77e8ac399e admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:08.550818 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906a810b-cd61-4875-a909-2e77e8ac399e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:08.552755 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906a810b-cd61-4875-a909-2e77e8ac399e admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:08.574872 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 793/1575] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:43:08 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:08.576107 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a0ba2-33e7-47ca-8367-1e2c7865cc48 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:08.576392 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a0ba2-33e7-47ca-8367-1e2c7865cc48 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:08.576660 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a0ba2-33e7-47ca-8367-1e2c7865cc48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:08.579684 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 783/1576] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:43:08 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:08.580629 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a810b-cd61-4875-a909-2e77e8ac399e admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:08.606976 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-136a0ba2-33e7-47ca-8367-1e2c7865cc48 None None] MFA Rules not processed for user `140b34e8949846549815df130d6ab172`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:08.608203 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-136a0ba2-33e7-47ca-8367-1e2c7865cc48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:08.642155 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a810b-cd61-4875-a909-2e77e8ac399e admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:08.647173 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e22adb7-629d-4ec8-a40c-c5d4fefc5100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:08.647708 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22adb7-629d-4ec8-a40c-c5d4fefc5100 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:08.648003 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22adb7-629d-4ec8-a40c-c5d4fefc5100 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:08.648276 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22adb7-629d-4ec8-a40c-c5d4fefc5100 admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:08.648896 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e22adb7-629d-4ec8-a40c-c5d4fefc5100 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:08.650802 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e22adb7-629d-4ec8-a40c-c5d4fefc5100 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:08.672603 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 794/1577] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:08.673444 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a0ba2-33e7-47ca-8367-1e2c7865cc48 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:08.685242 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 784/1578] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:43:08 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:08.685242 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e22adb7-629d-4ec8-a40c-c5d4fefc5100 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:08.690869 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a0ba2-33e7-47ca-8367-1e2c7865cc48 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:08.750497 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc6342-f99c-4c48-859f-600bdab5d056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:08.751107 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e22adb7-629d-4ec8-a40c-c5d4fefc5100 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:08.751271 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc6342-f99c-4c48-859f-600bdab5d056 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:08.751397 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc6342-f99c-4c48-859f-600bdab5d056 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:08.751766 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc6342-f99c-4c48-859f-600bdab5d056 admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:08.752428 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc6342-f99c-4c48-859f-600bdab5d056 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:08.754344 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc6342-f99c-4c48-859f-600bdab5d056 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:08.755593 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d7b30-76a1-48c1-8b81-9abe78ba760a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:08.756215 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d7b30-76a1-48c1-8b81-9abe78ba760a admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:08.756489 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d7b30-76a1-48c1-8b81-9abe78ba760a admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:08.756753 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d7b30-76a1-48c1-8b81-9abe78ba760a admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:08.757363 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9d7b30-76a1-48c1-8b81-9abe78ba760a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:08.759238 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9d7b30-76a1-48c1-8b81-9abe78ba760a admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:08.779528 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 795/1579] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:43:08 2022] POST /identity/v3/users => generated 295 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:08.780641 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc6342-f99c-4c48-859f-600bdab5d056 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:08.786298 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:08.793531 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 785/1580] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:43:08 2022] POST /identity/v3/projects => generated 371 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:08.793531 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d7b30-76a1-48c1-8b81-9abe78ba760a admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:08.793784 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc6342-f99c-4c48-859f-600bdab5d056 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:08.833089 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:08.850883 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1ff14d-37dd-4aec-b396-26af83dd9d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:08.851499 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ff14d-37dd-4aec-b396-26af83dd9d18 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:08.851770 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ff14d-37dd-4aec-b396-26af83dd9d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:08.852080 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ff14d-37dd-4aec-b396-26af83dd9d18 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:08.852723 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1ff14d-37dd-4aec-b396-26af83dd9d18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:08.854342 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1ff14d-37dd-4aec-b396-26af83dd9d18 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:08.855244 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d7b30-76a1-48c1-8b81-9abe78ba760a admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:08.859868 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80288616-ada9-4551-8515-31386868a68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:08.860532 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80288616-ada9-4551-8515-31386868a68e admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:08.860821 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80288616-ada9-4551-8515-31386868a68e admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:08.861094 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80288616-ada9-4551-8515-31386868a68e admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:08.861739 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80288616-ada9-4551-8515-31386868a68e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:08.863371 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80288616-ada9-4551-8515-31386868a68e admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:08.864360 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 796/1581] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:08.864360 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1ff14d-37dd-4aec-b396-26af83dd9d18 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:08.875554 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1ff14d-37dd-4aec-b396-26af83dd9d18 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:08.876824 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 786/1582] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:08.877638 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80288616-ada9-4551-8515-31386868a68e admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:08.882968 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d6fe5-5c96-4ffd-a057-5846d9894f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:08.883555 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d6fe5-5c96-4ffd-a057-5846d9894f94 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:08.883838 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d6fe5-5c96-4ffd-a057-5846d9894f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:08.884156 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d6fe5-5c96-4ffd-a057-5846d9894f94 admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:08.884768 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5d6fe5-5c96-4ffd-a057-5846d9894f94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:08.887225 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5d6fe5-5c96-4ffd-a057-5846d9894f94 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:08.887577 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80288616-ada9-4551-8515-31386868a68e admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:08.892085 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cd1745-1770-43cf-ba8f-fe0466b18ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:08.892660 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd1745-1770-43cf-ba8f-fe0466b18ed8 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:08.892940 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd1745-1770-43cf-ba8f-fe0466b18ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:08.893214 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd1745-1770-43cf-ba8f-fe0466b18ed8 admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:08.893823 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cd1745-1770-43cf-ba8f-fe0466b18ed8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:08.895725 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cd1745-1770-43cf-ba8f-fe0466b18ed8 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:08.909199 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 797/1583] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:43:08 2022] POST /identity/v3/users => generated 289 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:08.910299 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d6fe5-5c96-4ffd-a057-5846d9894f94 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:08.920566 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d6fe5-5c96-4ffd-a057-5846d9894f94 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:08.920811 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 787/1584] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:43:08 2022] POST /identity/v3/users => generated 283 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:08.922216 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427049ff-46b2-4c6d-83e2-24fa6b062883 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:08.922504 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427049ff-46b2-4c6d-83e2-24fa6b062883 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:08.922845 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427049ff-46b2-4c6d-83e2-24fa6b062883 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:08.924301 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 788/1585] 10.222.0.26 () {54 vars in 962 bytes} [Wed Aug 17 23:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:43:08.929220 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427049ff-46b2-4c6d-83e2-24fa6b062883 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:08.931029 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57661440-3a1c-4172-b36c-0c6ce2775143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:08.931583 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57661440-3a1c-4172-b36c-0c6ce2775143 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:08.931925 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57661440-3a1c-4172-b36c-0c6ce2775143 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:08.932840 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57661440-3a1c-4172-b36c-0c6ce2775143 admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:08.933564 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57661440-3a1c-4172-b36c-0c6ce2775143 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:08.935712 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57661440-3a1c-4172-b36c-0c6ce2775143 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:08.944214 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427049ff-46b2-4c6d-83e2-24fa6b062883 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:08.949760 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc78c49-70c2-4b05-8e01-686108da83f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:08.950358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc78c49-70c2-4b05-8e01-686108da83f7 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:08.950691 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc78c49-70c2-4b05-8e01-686108da83f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:08.950958 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc78c49-70c2-4b05-8e01-686108da83f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:08.951672 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc78c49-70c2-4b05-8e01-686108da83f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:08.953381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc78c49-70c2-4b05-8e01-686108da83f7 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:08.959124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 798/1586] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:43:08 2022] POST /identity/v3/users => generated 287 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:08.959921 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57661440-3a1c-4172-b36c-0c6ce2775143 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:08.967173 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 789/1587] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:08.967173 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc78c49-70c2-4b05-8e01-686108da83f7 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:08.971496 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57661440-3a1c-4172-b36c-0c6ce2775143 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:08.976252 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e13d2d-bd79-4861-b84a-efaad59ade36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:08.976489 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc78c49-70c2-4b05-8e01-686108da83f7 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:08.976838 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e13d2d-bd79-4861-b84a-efaad59ade36 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:08.977117 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e13d2d-bd79-4861-b84a-efaad59ade36 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:08.977388 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e13d2d-bd79-4861-b84a-efaad59ade36 admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:08.978007 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e13d2d-bd79-4861-b84a-efaad59ade36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:08.979985 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e13d2d-bd79-4861-b84a-efaad59ade36 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:08.981106 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798377cf-f49b-4d7b-be6b-387dd8efe961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:08.981772 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798377cf-f49b-4d7b-be6b-387dd8efe961 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:08.982100 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798377cf-f49b-4d7b-be6b-387dd8efe961 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:08.982358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798377cf-f49b-4d7b-be6b-387dd8efe961 admin admin] PATH_INFO: `/v3/projects/258851d93c2d466a85150116d68441b6/users/21e06d9bb5a240459ef3298a990a42f2/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:08.997905 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798377cf-f49b-4d7b-be6b-387dd8efe961 admin admin] RBAC: Authorizing `identity:create_grant(project_id=258851d93c2d466a85150116d68441b6, user_id=21e06d9bb5a240459ef3298a990a42f2, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:09.000604 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798377cf-f49b-4d7b-be6b-387dd8efe961 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:09.004895 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 799/1588] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:43:08 2022] POST /identity/v3/users => generated 286 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:09.004895 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e13d2d-bd79-4861-b84a-efaad59ade36 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:09.013965 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e13d2d-bd79-4861-b84a-efaad59ade36 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:09.016971 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 790/1589] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:43:08 2022] PUT /identity/v3/projects/258851d93c2d466a85150116d68441b6/users/21e06d9bb5a240459ef3298a990a42f2/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:09.017768 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798377cf-f49b-4d7b-be6b-387dd8efe961 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:09.072248 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5595b-d468-4666-9436-9bc03d3afdf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:09.073429 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5595b-d468-4666-9436-9bc03d3afdf8 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:09.073718 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5595b-d468-4666-9436-9bc03d3afdf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:09.073993 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5595b-d468-4666-9436-9bc03d3afdf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:09.074636 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e5595b-d468-4666-9436-9bc03d3afdf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:09.076271 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e5595b-d468-4666-9436-9bc03d3afdf8 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:09.079762 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798377cf-f49b-4d7b-be6b-387dd8efe961 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:09.084417 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7594b5a-f095-4463-b3c8-d6c71f4b7939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:09.085016 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7594b5a-f095-4463-b3c8-d6c71f4b7939 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:09.085302 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7594b5a-f095-4463-b3c8-d6c71f4b7939 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:09.085575 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7594b5a-f095-4463-b3c8-d6c71f4b7939 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:09.086217 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7594b5a-f095-4463-b3c8-d6c71f4b7939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:09.086335 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 800/1590] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:09.087026 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5595b-d468-4666-9436-9bc03d3afdf8 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:09.088057 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7594b5a-f095-4463-b3c8-d6c71f4b7939 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:09.096504 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5595b-d468-4666-9436-9bc03d3afdf8 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:09.098682 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 791/1591] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:09.099864 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b99470-cd25-4fda-85f8-172376f21076 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:09.100183 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b99470-cd25-4fda-85f8-172376f21076 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:09.100458 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b99470-cd25-4fda-85f8-172376f21076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:09.101291 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0810420-a1fc-4733-a9ba-85fc5756f818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:09.101852 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0810420-a1fc-4733-a9ba-85fc5756f818 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:09.102133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0810420-a1fc-4733-a9ba-85fc5756f818 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:09.102406 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0810420-a1fc-4733-a9ba-85fc5756f818 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:09.103052 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0810420-a1fc-4733-a9ba-85fc5756f818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:09.104726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0810420-a1fc-4733-a9ba-85fc5756f818 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:09.115581 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 801/1592] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:09.116403 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0810420-a1fc-4733-a9ba-85fc5756f818 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:09.126120 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0810420-a1fc-4733-a9ba-85fc5756f818 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:09.130660 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c9d2c-5be7-409a-9a2f-ce53b01d820c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:09.131234 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c9d2c-5be7-409a-9a2f-ce53b01d820c admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:09.131513 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c9d2c-5be7-409a-9a2f-ce53b01d820c admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:09.131788 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c9d2c-5be7-409a-9a2f-ce53b01d820c admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:09.132450 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c9d2c-5be7-409a-9a2f-ce53b01d820c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:09.134015 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c9d2c-5be7-409a-9a2f-ce53b01d820c admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:09.138837 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-79b99470-cd25-4fda-85f8-172376f21076 None None] MFA Rules not processed for user `ec1dcbe43b0749c3bcc4af9100d771d1`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:09.140075 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-79b99470-cd25-4fda-85f8-172376f21076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:09.144343 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 802/1593] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:09.144986 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c9d2c-5be7-409a-9a2f-ce53b01d820c admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:09.156456 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c9d2c-5be7-409a-9a2f-ce53b01d820c admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:09.161841 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607d725-a157-4c9d-9891-b464246a68d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:09.162367 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607d725-a157-4c9d-9891-b464246a68d7 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:09.162667 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607d725-a157-4c9d-9891-b464246a68d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:09.162931 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607d725-a157-4c9d-9891-b464246a68d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:09.163596 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f607d725-a157-4c9d-9891-b464246a68d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:09.165252 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f607d725-a157-4c9d-9891-b464246a68d7 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:09.176210 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 803/1594] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:09.176858 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607d725-a157-4c9d-9891-b464246a68d7 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:09.186825 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607d725-a157-4c9d-9891-b464246a68d7 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:09.191276 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42885e60-09bb-4668-95b3-5686bad79b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:09.191837 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42885e60-09bb-4668-95b3-5686bad79b55 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:09.192146 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42885e60-09bb-4668-95b3-5686bad79b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:09.192417 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42885e60-09bb-4668-95b3-5686bad79b55 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:09.193039 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42885e60-09bb-4668-95b3-5686bad79b55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:09.194582 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42885e60-09bb-4668-95b3-5686bad79b55 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:09.196583 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:09.202321 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:09.206738 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 804/1595] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:09.207762 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251b4ec-7308-400d-9b85-bae22481dfad None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:09.208066 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251b4ec-7308-400d-9b85-bae22481dfad None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:09.208337 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251b4ec-7308-400d-9b85-bae22481dfad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:09.219257 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 792/1596] 10.222.0.26 () {58 vars in 1059 bytes} [Wed Aug 17 23:43:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:09.220086 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b99470-cd25-4fda-85f8-172376f21076 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:09.226669 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-5251b4ec-7308-400d-9b85-bae22481dfad None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:09.227860 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-5251b4ec-7308-400d-9b85-bae22481dfad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:09.230514 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b99470-cd25-4fda-85f8-172376f21076 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:09.234952 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50593bf-1560-47ac-aec2-b3d814d8c373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:09.235584 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50593bf-1560-47ac-aec2-b3d814d8c373 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:09.235858 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50593bf-1560-47ac-aec2-b3d814d8c373 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:09.235994 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 805/1597] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:09.236165 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50593bf-1560-47ac-aec2-b3d814d8c373 admin admin] PATH_INFO: `/v3/projects/dbc0fe79b67f4446b1a4c36c068f9b00/users/c67dec924b7e4f0aa0874e6e21879889/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:09.239884 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251b4ec-7308-400d-9b85-bae22481dfad None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:09.249599 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251b4ec-7308-400d-9b85-bae22481dfad None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:09.251547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50593bf-1560-47ac-aec2-b3d814d8c373 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbc0fe79b67f4446b1a4c36c068f9b00, user_id=c67dec924b7e4f0aa0874e6e21879889, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:09.254101 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd51ab6-6ed1-4037-93f6-1c6c49f927cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:09.254264 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50593bf-1560-47ac-aec2-b3d814d8c373 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:09.254688 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd51ab6-6ed1-4037-93f6-1c6c49f927cb admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:09.255081 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd51ab6-6ed1-4037-93f6-1c6c49f927cb admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:09.255357 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd51ab6-6ed1-4037-93f6-1c6c49f927cb admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:09.256030 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd51ab6-6ed1-4037-93f6-1c6c49f927cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:09.257636 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd51ab6-6ed1-4037-93f6-1c6c49f927cb admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:09.267270 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 793/1598] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:43:09 2022] PUT /identity/v3/projects/dbc0fe79b67f4446b1a4c36c068f9b00/users/c67dec924b7e4f0aa0874e6e21879889/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:09.270764 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 806/1599] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:09.270764 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd51ab6-6ed1-4037-93f6-1c6c49f927cb admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:09.271703 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50593bf-1560-47ac-aec2-b3d814d8c373 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:09.279903 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd51ab6-6ed1-4037-93f6-1c6c49f927cb admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:09.333461 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50593bf-1560-47ac-aec2-b3d814d8c373 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:09.337625 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b496ca-25dc-4330-95d5-1d88fe685783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:09.337975 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47833c63-e287-4e10-9cde-eaa352af8eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:09.338267 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b496ca-25dc-4330-95d5-1d88fe685783 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:09.339113 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b496ca-25dc-4330-95d5-1d88fe685783 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:09.339222 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47833c63-e287-4e10-9cde-eaa352af8eb9 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:09.339391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47833c63-e287-4e10-9cde-eaa352af8eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:09.339504 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b496ca-25dc-4330-95d5-1d88fe685783 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:09.339655 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47833c63-e287-4e10-9cde-eaa352af8eb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:09.340062 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b496ca-25dc-4330-95d5-1d88fe685783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:09.340306 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47833c63-e287-4e10-9cde-eaa352af8eb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:09.341730 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b496ca-25dc-4330-95d5-1d88fe685783 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:09.341899 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47833c63-e287-4e10-9cde-eaa352af8eb9 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:09.351463 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 794/1600] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:09.351788 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 807/1601] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:09.352795 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b496ca-25dc-4330-95d5-1d88fe685783 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:09.355528 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47833c63-e287-4e10-9cde-eaa352af8eb9 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:09.362672 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b496ca-25dc-4330-95d5-1d88fe685783 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:09.364842 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47833c63-e287-4e10-9cde-eaa352af8eb9 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:09.367666 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b06b9-449e-4e87-b84c-f10cbd3f4331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:09.368286 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b06b9-449e-4e87-b84c-f10cbd3f4331 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:09.368571 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b06b9-449e-4e87-b84c-f10cbd3f4331 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:09.368844 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b06b9-449e-4e87-b84c-f10cbd3f4331 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:09.369478 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43b06b9-449e-4e87-b84c-f10cbd3f4331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:09.369609 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1552118-cc48-4179-989e-07d7a6a77653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:09.370199 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1552118-cc48-4179-989e-07d7a6a77653 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:09.370483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1552118-cc48-4179-989e-07d7a6a77653 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:09.370758 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1552118-cc48-4179-989e-07d7a6a77653 admin admin] PATH_INFO: `/v3/projects/d9574c6bc0ea4f78a0145fc2764a6fa9/users/2da686247e7f490ab7924d0456da60d5/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:09.371060 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43b06b9-449e-4e87-b84c-f10cbd3f4331 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:09.381454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 808/1602] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:09.382307 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b06b9-449e-4e87-b84c-f10cbd3f4331 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:09.388963 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1552118-cc48-4179-989e-07d7a6a77653 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9574c6bc0ea4f78a0145fc2764a6fa9, user_id=2da686247e7f490ab7924d0456da60d5, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:09.391685 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1552118-cc48-4179-989e-07d7a6a77653 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:09.405168 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 795/1603] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:43:09 2022] PUT /identity/v3/projects/d9574c6bc0ea4f78a0145fc2764a6fa9/users/2da686247e7f490ab7924d0456da60d5/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:09.405976 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1552118-cc48-4179-989e-07d7a6a77653 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:09.453830 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b06b9-449e-4e87-b84c-f10cbd3f4331 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:09.462915 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1552118-cc48-4179-989e-07d7a6a77653 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:09.467451 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c876cca2-ae21-495a-a3e1-d256326eacab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:09.468087 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c876cca2-ae21-495a-a3e1-d256326eacab service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:09.468370 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c876cca2-ae21-495a-a3e1-d256326eacab service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:09.468641 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c876cca2-ae21-495a-a3e1-d256326eacab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:09.509500 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:09.518955 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c2887-9fe8-4003-b3fd-07659ffdccc2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:09.519560 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c2887-9fe8-4003-b3fd-07659ffdccc2 service nova] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:09.519836 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c2887-9fe8-4003-b3fd-07659ffdccc2 service nova] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:09.520141 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c2887-9fe8-4003-b3fd-07659ffdccc2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:09.527605 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:09.531280 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c876cca2-ae21-495a-a3e1-d256326eacab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:09.533374 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c876cca2-ae21-495a-a3e1-d256326eacab service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:09.543797 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 796/1604] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:09.545246 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51836d05-08e4-485a-a184-ef3ec6716339 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:09.545536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51836d05-08e4-485a-a184-ef3ec6716339 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:09.545814 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51836d05-08e4-485a-a184-ef3ec6716339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:09.564827 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-51836d05-08e4-485a-a184-ef3ec6716339 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:09.566054 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-51836d05-08e4-485a-a184-ef3ec6716339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:09.577687 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 797/1605] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:09.577687 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51836d05-08e4-485a-a184-ef3ec6716339 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:09.586564 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51836d05-08e4-485a-a184-ef3ec6716339 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:09.591274 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaeee33-2a54-4cd6-893e-913db3914751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:09.591993 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaeee33-2a54-4cd6-893e-913db3914751 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:09.592234 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaeee33-2a54-4cd6-893e-913db3914751 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:09.592544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaeee33-2a54-4cd6-893e-913db3914751 admin admin] PATH_INFO: `/v3/projects/b7ebf323aa7c4874bd7d0e2f2873b591/users/7c852a7d720347be87d8205716dfc15f/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:09.594982 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c2887-9fe8-4003-b3fd-07659ffdccc2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:09.596854 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c2887-9fe8-4003-b3fd-07659ffdccc2 service nova] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:09.607462 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 809/1606] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:09.608414 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c2887-9fe8-4003-b3fd-07659ffdccc2 service nova] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:09.621013 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c2887-9fe8-4003-b3fd-07659ffdccc2 service nova] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:09.626613 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015eb567-3e33-46fb-bcab-fcd498b5a651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:09.627092 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaeee33-2a54-4cd6-893e-913db3914751 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7ebf323aa7c4874bd7d0e2f2873b591, user_id=7c852a7d720347be87d8205716dfc15f, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:09.627532 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015eb567-3e33-46fb-bcab-fcd498b5a651 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:09.628145 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015eb567-3e33-46fb-bcab-fcd498b5a651 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:09.628540 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015eb567-3e33-46fb-bcab-fcd498b5a651 admin admin] PATH_INFO: `/v3/projects/1f82fb6a4b3747edaf6a1b0cabee67c3/users/584c671b1a0a4c1aa792d4f5df7589c1/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:09.631414 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaeee33-2a54-4cd6-893e-913db3914751 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:09.652006 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 798/1607] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:43:09 2022] PUT /identity/v3/projects/b7ebf323aa7c4874bd7d0e2f2873b591/users/7c852a7d720347be87d8205716dfc15f/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:09.653188 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaeee33-2a54-4cd6-893e-913db3914751 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:09.653646 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015eb567-3e33-46fb-bcab-fcd498b5a651 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f82fb6a4b3747edaf6a1b0cabee67c3, user_id=584c671b1a0a4c1aa792d4f5df7589c1, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:09.655021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: WARNING py.warnings [None req-015eb567-3e33-46fb-bcab-fcd498b5a651 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 17 23:43:09.655021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: warnings.warn(msg) Aug 17 23:43:09.655021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: Aug 17 23:43:09.662557 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015eb567-3e33-46fb-bcab-fcd498b5a651 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:09.681272 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 810/1608] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:43:09 2022] PUT /identity/v3/projects/1f82fb6a4b3747edaf6a1b0cabee67c3/users/584c671b1a0a4c1aa792d4f5df7589c1/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:09.682953 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015eb567-3e33-46fb-bcab-fcd498b5a651 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:09.752441 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaeee33-2a54-4cd6-893e-913db3914751 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:09.804318 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015eb567-3e33-46fb-bcab-fcd498b5a651 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:09.809423 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7de1e82-76ab-45ea-bf82-8a6a80a36536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:09.810085 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de1e82-76ab-45ea-bf82-8a6a80a36536 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:09.810370 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de1e82-76ab-45ea-bf82-8a6a80a36536 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:09.810816 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de1e82-76ab-45ea-bf82-8a6a80a36536 admin admin] PATH_INFO: `/v3/projects/5393defca80f4e8e8bfbec7a4305d58d/users/2c5d003f5caf4437a0e6a892937ed693/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:09.828315 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41f3be-0b54-4741-8cdf-85925bec46f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:09.829010 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41f3be-0b54-4741-8cdf-85925bec46f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:09.829283 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41f3be-0b54-4741-8cdf-85925bec46f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:09.829551 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41f3be-0b54-4741-8cdf-85925bec46f8 admin admin] PATH_INFO: `/v3/projects/8069466b28f443ea8ab87ab598c1ede2/users/bfc475462cf04deb8b1a1c96665d9ea7/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:09.837772 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7de1e82-76ab-45ea-bf82-8a6a80a36536 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5393defca80f4e8e8bfbec7a4305d58d, user_id=2c5d003f5caf4437a0e6a892937ed693, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:09.841650 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7de1e82-76ab-45ea-bf82-8a6a80a36536 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:09.845926 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f41f3be-0b54-4741-8cdf-85925bec46f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8069466b28f443ea8ab87ab598c1ede2, user_id=bfc475462cf04deb8b1a1c96665d9ea7, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:09.850337 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f41f3be-0b54-4741-8cdf-85925bec46f8 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:09.858661 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 811/1609] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:43:09 2022] PUT /identity/v3/projects/5393defca80f4e8e8bfbec7a4305d58d/users/2c5d003f5caf4437a0e6a892937ed693/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:09.860009 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e29f4f-a75c-4c9a-89fa-658831210464 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:09.860841 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e29f4f-a75c-4c9a-89fa-658831210464 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:09.860841 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e29f4f-a75c-4c9a-89fa-658831210464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:09.875733 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 799/1610] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:43:09 2022] PUT /identity/v3/projects/8069466b28f443ea8ab87ab598c1ede2/users/bfc475462cf04deb8b1a1c96665d9ea7/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:09.888077 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41f3be-0b54-4741-8cdf-85925bec46f8 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:09.888340 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-80e29f4f-a75c-4c9a-89fa-658831210464 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:09.889812 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-80e29f4f-a75c-4c9a-89fa-658831210464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:09.889932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-3f41f3be-0b54-4741-8cdf-85925bec46f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:09.970005 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:09.971702 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41f3be-0b54-4741-8cdf-85925bec46f8 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:09.978111 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1636529-07cf-4f28-ab4a-cc287eea4cda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:09.979847 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1636529-07cf-4f28-ab4a-cc287eea4cda service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:09.980422 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1636529-07cf-4f28-ab4a-cc287eea4cda service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:09.981316 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1636529-07cf-4f28-ab4a-cc287eea4cda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:09.985801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 812/1611] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:09.985801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852266f-6af4-4ba9-a000-0dcb367033c4 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:09.985801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852266f-6af4-4ba9-a000-0dcb367033c4 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:09.985801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852266f-6af4-4ba9-a000-0dcb367033c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:10.002439 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:10.008814 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1636529-07cf-4f28-ab4a-cc287eea4cda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:10.011316 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1636529-07cf-4f28-ab4a-cc287eea4cda service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:10.012066 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-1852266f-6af4-4ba9-a000-0dcb367033c4 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:10.013659 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-1852266f-6af4-4ba9-a000-0dcb367033c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:10.023811 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 813/1612] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:10.025188 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6116a9f-133b-4069-93a7-27f3ba1e7504 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:10.025475 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6116a9f-133b-4069-93a7-27f3ba1e7504 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:10.025751 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6116a9f-133b-4069-93a7-27f3ba1e7504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:10.027686 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 800/1613] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:10.029012 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de08ab02-f038-41b4-bdb8-3dd904108713 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:10.029338 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de08ab02-f038-41b4-bdb8-3dd904108713 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:10.029612 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de08ab02-f038-41b4-bdb8-3dd904108713 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:10.031455 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 801/1614] 10.222.0.26 () {54 vars in 912 bytes} [Wed Aug 17 23:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:43:10.032548 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ad4cb-6a6c-4f12-a74d-e34f006abf12 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:10.032826 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ad4cb-6a6c-4f12-a74d-e34f006abf12 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:10.033095 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ad4cb-6a6c-4f12-a74d-e34f006abf12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:10.069990 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-511ad4cb-6a6c-4f12-a74d-e34f006abf12 None None] MFA Rules not processed for user `2c5d003f5caf4437a0e6a892937ed693`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:10.071438 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-511ad4cb-6a6c-4f12-a74d-e34f006abf12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:10.071583 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-d6116a9f-133b-4069-93a7-27f3ba1e7504 None None] MFA Rules not processed for user `584c671b1a0a4c1aa792d4f5df7589c1`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:10.073021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-d6116a9f-133b-4069-93a7-27f3ba1e7504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:10.166336 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 802/1615] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:10.168713 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e648fa5-ff22-4c9d-83c4-8604d8ecdcb5 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:10.169289 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e648fa5-ff22-4c9d-83c4-8604d8ecdcb5 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:10.169576 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e648fa5-ff22-4c9d-83c4-8604d8ecdcb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:10.172202 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 814/1616] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:10.173500 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6116a9f-133b-4069-93a7-27f3ba1e7504 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:10.189400 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6116a9f-133b-4069-93a7-27f3ba1e7504 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:10.194448 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e925bb2-a21b-45d0-a819-5dab217f4ff8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:10.196327 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e925bb2-a21b-45d0-a819-5dab217f4ff8 service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:10.196696 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e925bb2-a21b-45d0-a819-5dab217f4ff8 service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:10.197017 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e925bb2-a21b-45d0-a819-5dab217f4ff8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:10.205558 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-0e648fa5-ff22-4c9d-83c4-8604d8ecdcb5 None None] MFA Rules not processed for user `bfc475462cf04deb8b1a1c96665d9ea7`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:10.207818 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-0e648fa5-ff22-4c9d-83c4-8604d8ecdcb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:10.210027 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e925bb2-a21b-45d0-a819-5dab217f4ff8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:10.212836 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e925bb2-a21b-45d0-a819-5dab217f4ff8 service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:10.226909 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 815/1617] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:10.228933 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e925bb2-a21b-45d0-a819-5dab217f4ff8 service neutron] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:10.242867 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e925bb2-a21b-45d0-a819-5dab217f4ff8 service neutron] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:10.249210 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd4282-50d8-4e67-ba59-6f45c7afb2a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:10.250188 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd4282-50d8-4e67-ba59-6f45c7afb2a9 service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:10.250664 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd4282-50d8-4e67-ba59-6f45c7afb2a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:10.250918 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd4282-50d8-4e67-ba59-6f45c7afb2a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:10.264983 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd4282-50d8-4e67-ba59-6f45c7afb2a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:10.267947 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd4282-50d8-4e67-ba59-6f45c7afb2a9 service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:10.313629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 816/1618] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:10.315243 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1fbd5-3665-4642-93e9-00d79de17c0c None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:10.318746 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1fbd5-3665-4642-93e9-00d79de17c0c None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:10.319040 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1fbd5-3665-4642-93e9-00d79de17c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:10.336356 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 803/1619] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:10.342560 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e648fa5-ff22-4c9d-83c4-8604d8ecdcb5 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:10.369672 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e648fa5-ff22-4c9d-83c4-8604d8ecdcb5 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:10.374145 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-83d1fbd5-3665-4642-93e9-00d79de17c0c None None] MFA Rules not processed for user `ec1dcbe43b0749c3bcc4af9100d771d1`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:10.375386 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-83d1fbd5-3665-4642-93e9-00d79de17c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:10.383113 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d1f6f-1b98-45a0-bf7d-9cdb800bf170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:10.383911 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d1f6f-1b98-45a0-bf7d-9cdb800bf170 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:10.384493 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d1f6f-1b98-45a0-bf7d-9cdb800bf170 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:10.384761 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d1f6f-1b98-45a0-bf7d-9cdb800bf170 admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:10.385698 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340d1f6f-1b98-45a0-bf7d-9cdb800bf170 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:10.390391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340d1f6f-1b98-45a0-bf7d-9cdb800bf170 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:10.466528 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 804/1620] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:43:10 2022] POST /identity/v3/projects => generated 373 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:10.468104 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cdf35-d71e-4ecc-ae9a-2e9f53800c77 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:10.468104 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cdf35-d71e-4ecc-ae9a-2e9f53800c77 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:10.468104 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cdf35-d71e-4ecc-ae9a-2e9f53800c77 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:10.474082 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 805/1621] 10.222.0.26 () {54 vars in 962 bytes} [Wed Aug 17 23:43:10 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:43:10.482738 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1cdf35-d71e-4ecc-ae9a-2e9f53800c77 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:10.512868 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 817/1622] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Aug 17 23:43:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:10.513799 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d1fbd5-3665-4642-93e9-00d79de17c0c None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:10.527118 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d1fbd5-3665-4642-93e9-00d79de17c0c None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:10.600159 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:10.607887 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:10.627182 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a38dd7-32c0-498b-b057-3b5684e0fd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:10.628285 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a38dd7-32c0-498b-b057-3b5684e0fd5b admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:10.629531 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a38dd7-32c0-498b-b057-3b5684e0fd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:10.629531 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a38dd7-32c0-498b-b057-3b5684e0fd5b admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:10.630219 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a38dd7-32c0-498b-b057-3b5684e0fd5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:10.632987 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a38dd7-32c0-498b-b057-3b5684e0fd5b admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:10.634889 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1cdf35-d71e-4ecc-ae9a-2e9f53800c77 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:10.648394 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0d93ee-b332-48b7-b7d4-664dc69e171e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:10.649186 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d93ee-b332-48b7-b7d4-664dc69e171e admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:10.649474 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d93ee-b332-48b7-b7d4-664dc69e171e admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:10.649900 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d93ee-b332-48b7-b7d4-664dc69e171e admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:10.656557 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0d93ee-b332-48b7-b7d4-664dc69e171e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:10.663199 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0d93ee-b332-48b7-b7d4-664dc69e171e admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:10.667325 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 818/1623] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:43:10 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:10.669940 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82561a2-6c1b-46ea-ae35-8007be4d2989 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:10.670391 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82561a2-6c1b-46ea-ae35-8007be4d2989 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:10.670871 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82561a2-6c1b-46ea-ae35-8007be4d2989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:10.700972 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-b82561a2-6c1b-46ea-ae35-8007be4d2989 None None] MFA Rules not processed for user `ec1dcbe43b0749c3bcc4af9100d771d1`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:10.702625 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-b82561a2-6c1b-46ea-ae35-8007be4d2989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:10.727666 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 806/1624] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:43:10 2022] POST /identity/v3/projects => generated 365 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:10.728583 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0d93ee-b332-48b7-b7d4-664dc69e171e admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:10.843335 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0d93ee-b332-48b7-b7d4-664dc69e171e admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:10.847731 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 819/1625] 10.222.0.26 () {58 vars in 1059 bytes} [Wed Aug 17 23:43:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:10.848978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b5e57-272c-44f1-abb9-4c5a0b24b30b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:10.849672 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b5e57-272c-44f1-abb9-4c5a0b24b30b service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:10.850009 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b5e57-272c-44f1-abb9-4c5a0b24b30b service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:10.850343 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b5e57-272c-44f1-abb9-4c5a0b24b30b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:10.851723 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82561a2-6c1b-46ea-ae35-8007be4d2989 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:10.876611 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82561a2-6c1b-46ea-ae35-8007be4d2989 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:11.020488 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896b5e57-272c-44f1-abb9-4c5a0b24b30b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:11.023429 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896b5e57-272c-44f1-abb9-4c5a0b24b30b service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:11.038474 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33024e85-a4f9-44bf-869b-e77e2de096bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:11.039098 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33024e85-a4f9-44bf-869b-e77e2de096bd admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:11.039381 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33024e85-a4f9-44bf-869b-e77e2de096bd admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:11.039650 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33024e85-a4f9-44bf-869b-e77e2de096bd admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:11.040340 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33024e85-a4f9-44bf-869b-e77e2de096bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:11.040907 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 807/1626] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:11.041986 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33024e85-a4f9-44bf-869b-e77e2de096bd admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:11.043237 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b5e57-272c-44f1-abb9-4c5a0b24b30b service cinder] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:11.055233 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b5e57-272c-44f1-abb9-4c5a0b24b30b service cinder] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:11.062261 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a1d7e-5c31-4476-845a-44e5854d91ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:11.063081 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a1d7e-5c31-4476-845a-44e5854d91ec admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:11.063469 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a1d7e-5c31-4476-845a-44e5854d91ec admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:11.063987 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a1d7e-5c31-4476-845a-44e5854d91ec admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:11.065050 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05a1d7e-5c31-4476-845a-44e5854d91ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:11.067286 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05a1d7e-5c31-4476-845a-44e5854d91ec admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:11.068155 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 820/1627] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:10 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:11.073355 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33024e85-a4f9-44bf-869b-e77e2de096bd admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:11.099063 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 808/1628] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:43:11 2022] POST /identity/v3/users => generated 289 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:11.099063 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a1d7e-5c31-4476-845a-44e5854d91ec admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:11.107396 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33024e85-a4f9-44bf-869b-e77e2de096bd admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:11.110470 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a1d7e-5c31-4476-845a-44e5854d91ec admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:11.113395 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e4ea52-b1bd-4f76-8657-1e1f23905234 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:11.114018 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4ea52-b1bd-4f76-8657-1e1f23905234 service nova] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:11.114300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4ea52-b1bd-4f76-8657-1e1f23905234 service nova] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:11.114575 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4ea52-b1bd-4f76-8657-1e1f23905234 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:11.118298 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea4e97-9631-4d31-a1be-6f0d5e25119a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:11.119354 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea4e97-9631-4d31-a1be-6f0d5e25119a admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:11.120468 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea4e97-9631-4d31-a1be-6f0d5e25119a admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:11.120865 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea4e97-9631-4d31-a1be-6f0d5e25119a admin admin] PATH_INFO: `/v3/projects/e8000d42754b4f92be5b1de5201b225f/users/aad952a8d08c4d9b81a80c3c98a8fa82/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:11.148952 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ea4e97-9631-4d31-a1be-6f0d5e25119a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8000d42754b4f92be5b1de5201b225f, user_id=aad952a8d08c4d9b81a80c3c98a8fa82, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:11.152321 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ea4e97-9631-4d31-a1be-6f0d5e25119a admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:11.172416 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 809/1629] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:43:11 2022] PUT /identity/v3/projects/e8000d42754b4f92be5b1de5201b225f/users/aad952a8d08c4d9b81a80c3c98a8fa82/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:11.173350 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea4e97-9631-4d31-a1be-6f0d5e25119a admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:11.237387 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e4ea52-b1bd-4f76-8657-1e1f23905234 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:11.238950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e4ea52-b1bd-4f76-8657-1e1f23905234 service nova] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:11.269315 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:11.272915 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 821/1630] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:11 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:11.273324 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e4ea52-b1bd-4f76-8657-1e1f23905234 service nova] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:11.298444 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:11.304016 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e4ea52-b1bd-4f76-8657-1e1f23905234 service nova] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:11.315855 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea4e97-9631-4d31-a1be-6f0d5e25119a admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:11.318897 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee481f78-0ec1-42ff-907c-526e15d5c2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:11.319527 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee481f78-0ec1-42ff-907c-526e15d5c2ca admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:11.319806 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee481f78-0ec1-42ff-907c-526e15d5c2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:11.320100 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee481f78-0ec1-42ff-907c-526e15d5c2ca admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:11.320880 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee481f78-0ec1-42ff-907c-526e15d5c2ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:11.321677 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab44e8e1-4581-4a8c-9d1e-cd359e242ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:11.322339 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee481f78-0ec1-42ff-907c-526e15d5c2ca admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:11.322470 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44e8e1-4581-4a8c-9d1e-cd359e242ffd admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:11.323536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44e8e1-4581-4a8c-9d1e-cd359e242ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:11.323853 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44e8e1-4581-4a8c-9d1e-cd359e242ffd admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:11.325149 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab44e8e1-4581-4a8c-9d1e-cd359e242ffd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:11.327221 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab44e8e1-4581-4a8c-9d1e-cd359e242ffd admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:11.344292 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 810/1631] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:11.348396 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab44e8e1-4581-4a8c-9d1e-cd359e242ffd admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:11.351908 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 822/1632] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:11.355931 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee481f78-0ec1-42ff-907c-526e15d5c2ca admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:11.364526 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab44e8e1-4581-4a8c-9d1e-cd359e242ffd admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:11.374836 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc34e953-1f01-45e7-9a5f-c0bbedfbcd57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:11.375516 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34e953-1f01-45e7-9a5f-c0bbedfbcd57 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:11.375805 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34e953-1f01-45e7-9a5f-c0bbedfbcd57 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:11.376112 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34e953-1f01-45e7-9a5f-c0bbedfbcd57 admin admin] PATH_INFO: `/v3/projects/b956759927da4fe0b70c011b4d0718d5/users/4098d610971e462ab2c9e6d80b85355c/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:11.379494 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee481f78-0ec1-42ff-907c-526e15d5c2ca admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:11.394559 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c8f261-ff08-446f-b5b2-3f226784d62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:11.395530 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8f261-ff08-446f-b5b2-3f226784d62c admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:11.396130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8f261-ff08-446f-b5b2-3f226784d62c admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:11.396697 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8f261-ff08-446f-b5b2-3f226784d62c admin admin] PATH_INFO: `/v3/projects/e8000d42754b4f92be5b1de5201b225f/users/aad952a8d08c4d9b81a80c3c98a8fa82/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:11.398504 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc34e953-1f01-45e7-9a5f-c0bbedfbcd57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b956759927da4fe0b70c011b4d0718d5, user_id=4098d610971e462ab2c9e6d80b85355c, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:11.400369 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c8f261-ff08-446f-b5b2-3f226784d62c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8000d42754b4f92be5b1de5201b225f, user_id=aad952a8d08c4d9b81a80c3c98a8fa82, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:11.402675 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc34e953-1f01-45e7-9a5f-c0bbedfbcd57 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:11.408009 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c8f261-ff08-446f-b5b2-3f226784d62c admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:11.432700 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 811/1633] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:43:11 2022] PUT /identity/v3/projects/b956759927da4fe0b70c011b4d0718d5/users/4098d610971e462ab2c9e6d80b85355c/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:11.435585 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 823/1634] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:43:11 2022] PUT /identity/v3/projects/e8000d42754b4f92be5b1de5201b225f/users/aad952a8d08c4d9b81a80c3c98a8fa82/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:11.436660 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc34e953-1f01-45e7-9a5f-c0bbedfbcd57 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:11.447236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed0092-ddbc-44ed-a96b-19b6941e65f1 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:11.450036 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed0092-ddbc-44ed-a96b-19b6941e65f1 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:11.450513 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed0092-ddbc-44ed-a96b-19b6941e65f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:11.530271 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-c1ed0092-ddbc-44ed-a96b-19b6941e65f1 None None] MFA Rules not processed for user `aad952a8d08c4d9b81a80c3c98a8fa82`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:11.534632 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-c1ed0092-ddbc-44ed-a96b-19b6941e65f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:11.617622 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc34e953-1f01-45e7-9a5f-c0bbedfbcd57 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:11.745142 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 824/1635] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:11 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:11.751130 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4bfa5-9f19-4834-8ac3-e2ac6cc605a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:11.751935 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4bfa5-9f19-4834-8ac3-e2ac6cc605a0 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:11.752101 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4bfa5-9f19-4834-8ac3-e2ac6cc605a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:11.752430 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4bfa5-9f19-4834-8ac3-e2ac6cc605a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:11.753183 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b4bfa5-9f19-4834-8ac3-e2ac6cc605a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:11.754818 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b4bfa5-9f19-4834-8ac3-e2ac6cc605a0 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:11.758772 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed0092-ddbc-44ed-a96b-19b6941e65f1 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:11.798347 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 812/1636] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:11.799803 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e03e084-bb8f-4f1c-81bb-57dc71b761fa None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:11.802560 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e03e084-bb8f-4f1c-81bb-57dc71b761fa None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:11.802859 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e03e084-bb8f-4f1c-81bb-57dc71b761fa None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:11.806272 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 813/1637] 10.222.0.26 () {54 vars in 912 bytes} [Wed Aug 17 23:43:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:43:11.809079 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e03e084-bb8f-4f1c-81bb-57dc71b761fa None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:11.882115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e03e084-bb8f-4f1c-81bb-57dc71b761fa None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:11.882115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c593146e-a71d-4fc5-8a95-43f98d46f520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:11.882115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593146e-a71d-4fc5-8a95-43f98d46f520 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:11.882115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593146e-a71d-4fc5-8a95-43f98d46f520 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:11.883689 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593146e-a71d-4fc5-8a95-43f98d46f520 admin admin] PATH_INFO: `/v3/projects/b956759927da4fe0b70c011b4d0718d5/users/4098d610971e462ab2c9e6d80b85355c/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:11.883689 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c593146e-a71d-4fc5-8a95-43f98d46f520 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b956759927da4fe0b70c011b4d0718d5, user_id=4098d610971e462ab2c9e6d80b85355c, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:11.883689 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c593146e-a71d-4fc5-8a95-43f98d46f520 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:11.885685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed0092-ddbc-44ed-a96b-19b6941e65f1 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:11.891234 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 814/1638] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:43:11 2022] PUT /identity/v3/projects/b956759927da4fe0b70c011b4d0718d5/users/4098d610971e462ab2c9e6d80b85355c/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:11.891234 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58db28-4cad-4044-857d-f175dfa058cd None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:11.891648 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58db28-4cad-4044-857d-f175dfa058cd None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:11.891818 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58db28-4cad-4044-857d-f175dfa058cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:11.950837 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-be58db28-4cad-4044-857d-f175dfa058cd None None] MFA Rules not processed for user `ec1dcbe43b0749c3bcc4af9100d771d1`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:11.952441 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-be58db28-4cad-4044-857d-f175dfa058cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:12.018222 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f21f47-619e-4842-b10a-a1280465857e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:12.019136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f21f47-619e-4842-b10a-a1280465857e service cinder] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:12.019419 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f21f47-619e-4842-b10a-a1280465857e service cinder] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:12.019694 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f21f47-619e-4842-b10a-a1280465857e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:12.028690 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:12.080327 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:12.151568 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f21f47-619e-4842-b10a-a1280465857e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:12.154299 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f21f47-619e-4842-b10a-a1280465857e service cinder] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:12.161833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 815/1639] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Aug 17 23:43:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:12.164273 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604b4af-85e7-4ee1-92b4-6a935e3c2660 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:12.164642 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604b4af-85e7-4ee1-92b4-6a935e3c2660 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:12.164994 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604b4af-85e7-4ee1-92b4-6a935e3c2660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:12.184682 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 825/1640] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:11 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:12.233355 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f21f47-619e-4842-b10a-a1280465857e service cinder] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:12.233945 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-6604b4af-85e7-4ee1-92b4-6a935e3c2660 None None] MFA Rules not processed for user `4098d610971e462ab2c9e6d80b85355c`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:12.234805 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-6604b4af-85e7-4ee1-92b4-6a935e3c2660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:12.246498 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f21f47-619e-4842-b10a-a1280465857e service cinder] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:12.251569 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483fbbf-896d-4303-a7ff-1e92eadcf0fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:12.252510 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483fbbf-896d-4303-a7ff-1e92eadcf0fb service cinder] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:12.253008 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483fbbf-896d-4303-a7ff-1e92eadcf0fb service cinder] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:12.253525 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483fbbf-896d-4303-a7ff-1e92eadcf0fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:12.268192 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e483fbbf-896d-4303-a7ff-1e92eadcf0fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:12.270014 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e483fbbf-896d-4303-a7ff-1e92eadcf0fb service cinder] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:12.282584 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 826/1641] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:12.370944 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 816/1642] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:12 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:12.609468 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483fbbf-896d-4303-a7ff-1e92eadcf0fb service cinder] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:12.625923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483fbbf-896d-4303-a7ff-1e92eadcf0fb service cinder] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:12.638255 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed29d59-e6dd-4d9d-90e4-0139c403b374 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:12.643731 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed29d59-e6dd-4d9d-90e4-0139c403b374 service cinder] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:12.645564 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed29d59-e6dd-4d9d-90e4-0139c403b374 service cinder] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:12.646041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed29d59-e6dd-4d9d-90e4-0139c403b374 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:12.681241 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed29d59-e6dd-4d9d-90e4-0139c403b374 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:12.688726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed29d59-e6dd-4d9d-90e4-0139c403b374 service cinder] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:12.744387 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 827/1643] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:12 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:13.185482 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6604b4af-85e7-4ee1-92b4-6a935e3c2660 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:13.197237 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6604b4af-85e7-4ee1-92b4-6a935e3c2660 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:13.205351 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df34865-72e0-438e-a12e-f77505aee0a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:13.206188 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df34865-72e0-438e-a12e-f77505aee0a2 service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:13.206654 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df34865-72e0-438e-a12e-f77505aee0a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:13.207323 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df34865-72e0-438e-a12e-f77505aee0a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:13.328749 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df34865-72e0-438e-a12e-f77505aee0a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:13.330654 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df34865-72e0-438e-a12e-f77505aee0a2 service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:13.348088 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 817/1644] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:13.792631 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed29d59-e6dd-4d9d-90e4-0139c403b374 service cinder] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:13.885397 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed29d59-e6dd-4d9d-90e4-0139c403b374 service cinder] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:13.890917 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143b4cad-9939-49f4-9156-6e24913b8576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:13.892003 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b4cad-9939-49f4-9156-6e24913b8576 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:13.892404 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b4cad-9939-49f4-9156-6e24913b8576 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:13.892857 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b4cad-9939-49f4-9156-6e24913b8576 admin admin] PATH_INFO: `/v3/users/584c671b1a0a4c1aa792d4f5df7589c1` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:13.895476 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143b4cad-9939-49f4-9156-6e24913b8576 admin admin] RBAC: Authorizing `identity:delete_user(user_id=584c671b1a0a4c1aa792d4f5df7589c1)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:13.897285 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: WARNING py.warnings [None req-143b4cad-9939-49f4-9156-6e24913b8576 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 17 23:43:13.897285 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: warnings.warn(msg) Aug 17 23:43:13.897285 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: Aug 17 23:43:13.899087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143b4cad-9939-49f4-9156-6e24913b8576 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:13.991477 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-143b4cad-9939-49f4-9156-6e24913b8576 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '584c671b1a0a4c1aa792d4f5df7589c1'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:13.992378 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-143b4cad-9939-49f4-9156-6e24913b8576 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '584c671b1a0a4c1aa792d4f5df7589c1'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:13.993107 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-143b4cad-9939-49f4-9156-6e24913b8576 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '584c671b1a0a4c1aa792d4f5df7589c1'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:14.004053 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-143b4cad-9939-49f4-9156-6e24913b8576 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '584c671b1a0a4c1aa792d4f5df7589c1'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:14.034171 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-143b4cad-9939-49f4-9156-6e24913b8576 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '584c671b1a0a4c1aa792d4f5df7589c1'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:14.068322 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 828/1645] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:43:13 2022] DELETE /identity/v3/users/584c671b1a0a4c1aa792d4f5df7589c1 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:14.073942 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf5efc-ca42-47c9-8229-bb5c096f1876 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:14.074480 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf5efc-ca42-47c9-8229-bb5c096f1876 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:14.074861 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf5efc-ca42-47c9-8229-bb5c096f1876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:14.101364 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-4cdf5efc-ca42-47c9-8229-bb5c096f1876 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:14.102912 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-4cdf5efc-ca42-47c9-8229-bb5c096f1876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:14.191341 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:14.215357 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 818/1646] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:14.230525 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143b4cad-9939-49f4-9156-6e24913b8576 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:14.232391 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-143b4cad-9939-49f4-9156-6e24913b8576 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:14.275156 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:14.343224 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143b4cad-9939-49f4-9156-6e24913b8576 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:14.347639 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486c804-3750-4a41-a04c-76988bee46a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:14.349228 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486c804-3750-4a41-a04c-76988bee46a6 service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:14.350139 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486c804-3750-4a41-a04c-76988bee46a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:14.350757 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486c804-3750-4a41-a04c-76988bee46a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:14.366602 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9486c804-3750-4a41-a04c-76988bee46a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:14.368671 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9486c804-3750-4a41-a04c-76988bee46a6 service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:14.383118 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 829/1647] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:15.769858 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf5efc-ca42-47c9-8229-bb5c096f1876 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:15.773130 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-4cdf5efc-ca42-47c9-8229-bb5c096f1876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:15.823805 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf5efc-ca42-47c9-8229-bb5c096f1876 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:15.846852 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6467c6f-7ae0-4491-b316-503f50e07487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:15.849926 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6467c6f-7ae0-4491-b316-503f50e07487 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:15.850353 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6467c6f-7ae0-4491-b316-503f50e07487 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:15.851712 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6467c6f-7ae0-4491-b316-503f50e07487 admin admin] PATH_INFO: `/v3/projects/1f82fb6a4b3747edaf6a1b0cabee67c3` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:15.857316 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6467c6f-7ae0-4491-b316-503f50e07487 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f82fb6a4b3747edaf6a1b0cabee67c3)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:15.861241 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: WARNING py.warnings [None req-e6467c6f-7ae0-4491-b316-503f50e07487 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 17 23:43:15.861241 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: warnings.warn(msg) Aug 17 23:43:15.861241 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: Aug 17 23:43:15.865348 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6467c6f-7ae0-4491-b316-503f50e07487 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:16.054917 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-e6467c6f-7ae0-4491-b316-503f50e07487 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f82fb6a4b3747edaf6a1b0cabee67c3', 'request_id': 'req-e6467c6f-7ae0-4491-b316-503f50e07487'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:16.061945 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-e6467c6f-7ae0-4491-b316-503f50e07487 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f82fb6a4b3747edaf6a1b0cabee67c3', 'request_id': 'req-e6467c6f-7ae0-4491-b316-503f50e07487'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:16.072197 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-e6467c6f-7ae0-4491-b316-503f50e07487 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f82fb6a4b3747edaf6a1b0cabee67c3', 'request_id': 'req-e6467c6f-7ae0-4491-b316-503f50e07487'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:16.098817 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-e6467c6f-7ae0-4491-b316-503f50e07487 admin admin] The token cache is being invalidate because project 1f82fb6a4b3747edaf6a1b0cabee67c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:43:16.099152 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-e6467c6f-7ae0-4491-b316-503f50e07487 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:16.099925 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-e6467c6f-7ae0-4491-b316-503f50e07487 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:16.113642 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 819/1648] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:43:15 2022] DELETE /identity/v3/projects/1f82fb6a4b3747edaf6a1b0cabee67c3 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:16.114829 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486c804-3750-4a41-a04c-76988bee46a6 service neutron] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:16.118727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-9486c804-3750-4a41-a04c-76988bee46a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:16.139627 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6467c6f-7ae0-4491-b316-503f50e07487 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:16.144656 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-e6467c6f-7ae0-4491-b316-503f50e07487 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:16.154963 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486c804-3750-4a41-a04c-76988bee46a6 service neutron] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:16.320471 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6467c6f-7ae0-4491-b316-503f50e07487 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:16.326081 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec1b32-42e1-4014-abbc-f7d9fbc2a133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:16.326797 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec1b32-42e1-4014-abbc-f7d9fbc2a133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:16.327084 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec1b32-42e1-4014-abbc-f7d9fbc2a133 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:16.327408 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec1b32-42e1-4014-abbc-f7d9fbc2a133 admin admin] PATH_INFO: `/v3/users/2c5d003f5caf4437a0e6a892937ed693` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:16.329146 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec1b32-42e1-4014-abbc-f7d9fbc2a133 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c5d003f5caf4437a0e6a892937ed693)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:16.332276 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec1b32-42e1-4014-abbc-f7d9fbc2a133 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:16.337031 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274d60e-960a-4edf-ac75-1cdabd3e4ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:16.337752 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274d60e-960a-4edf-ac75-1cdabd3e4ec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:16.338039 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274d60e-960a-4edf-ac75-1cdabd3e4ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:16.339169 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274d60e-960a-4edf-ac75-1cdabd3e4ec8 admin admin] PATH_INFO: `/v3/users/aad952a8d08c4d9b81a80c3c98a8fa82` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:16.340569 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0274d60e-960a-4edf-ac75-1cdabd3e4ec8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aad952a8d08c4d9b81a80c3c98a8fa82)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:16.348932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0274d60e-960a-4edf-ac75-1cdabd3e4ec8 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:16.467014 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-7fec1b32-42e1-4014-abbc-f7d9fbc2a133 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c5d003f5caf4437a0e6a892937ed693'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:16.470775 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-7fec1b32-42e1-4014-abbc-f7d9fbc2a133 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c5d003f5caf4437a0e6a892937ed693'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:16.472343 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-7fec1b32-42e1-4014-abbc-f7d9fbc2a133 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c5d003f5caf4437a0e6a892937ed693'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:16.476698 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-0274d60e-960a-4edf-ac75-1cdabd3e4ec8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aad952a8d08c4d9b81a80c3c98a8fa82'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:16.478710 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-0274d60e-960a-4edf-ac75-1cdabd3e4ec8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aad952a8d08c4d9b81a80c3c98a8fa82'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:16.491368 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-7fec1b32-42e1-4014-abbc-f7d9fbc2a133 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c5d003f5caf4437a0e6a892937ed693'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:16.510747 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-7fec1b32-42e1-4014-abbc-f7d9fbc2a133 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c5d003f5caf4437a0e6a892937ed693'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:16.519822 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-0274d60e-960a-4edf-ac75-1cdabd3e4ec8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aad952a8d08c4d9b81a80c3c98a8fa82'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:16.545235 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-0274d60e-960a-4edf-ac75-1cdabd3e4ec8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aad952a8d08c4d9b81a80c3c98a8fa82'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:16.546618 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-0274d60e-960a-4edf-ac75-1cdabd3e4ec8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aad952a8d08c4d9b81a80c3c98a8fa82'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:16.563307 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:16.576410 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 830/1649] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:43:16 2022] DELETE /identity/v3/users/2c5d003f5caf4437a0e6a892937ed693 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:16.582049 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9007606a-7867-4b02-8d7b-443ebd3a4d03 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:16.586164 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 820/1650] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:43:16 2022] DELETE /identity/v3/users/aad952a8d08c4d9b81a80c3c98a8fa82 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:16.586752 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9007606a-7867-4b02-8d7b-443ebd3a4d03 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:16.587442 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9007606a-7867-4b02-8d7b-443ebd3a4d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:16.614518 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-9007606a-7867-4b02-8d7b-443ebd3a4d03 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:16.615864 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-9007606a-7867-4b02-8d7b-443ebd3a4d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:16.692623 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:16.742120 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 831/1651] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:16.800931 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274d60e-960a-4edf-ac75-1cdabd3e4ec8 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:16.804012 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-0274d60e-960a-4edf-ac75-1cdabd3e4ec8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:16.911949 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274d60e-960a-4edf-ac75-1cdabd3e4ec8 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:16.917422 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac5699-4321-44af-b01b-2018a3875899 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:16.918458 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac5699-4321-44af-b01b-2018a3875899 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:16.919006 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac5699-4321-44af-b01b-2018a3875899 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:16.922820 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac5699-4321-44af-b01b-2018a3875899 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:16.953291 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeac5699-4321-44af-b01b-2018a3875899 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:16.956931 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeac5699-4321-44af-b01b-2018a3875899 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:17.008096 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 821/1652] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:18.333507 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9007606a-7867-4b02-8d7b-443ebd3a4d03 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:18.335038 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-9007606a-7867-4b02-8d7b-443ebd3a4d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:18.357876 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9007606a-7867-4b02-8d7b-443ebd3a4d03 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:18.371556 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1aaa20-dfa7-40f7-89de-36214730556a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:18.372813 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1aaa20-dfa7-40f7-89de-36214730556a admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:18.373286 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1aaa20-dfa7-40f7-89de-36214730556a admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:18.373393 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1aaa20-dfa7-40f7-89de-36214730556a admin admin] PATH_INFO: `/v3/projects/e8000d42754b4f92be5b1de5201b225f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:18.375898 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1aaa20-dfa7-40f7-89de-36214730556a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8000d42754b4f92be5b1de5201b225f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:18.378279 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1aaa20-dfa7-40f7-89de-36214730556a admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:18.489678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-9c1aaa20-dfa7-40f7-89de-36214730556a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8000d42754b4f92be5b1de5201b225f', 'request_id': 'req-9c1aaa20-dfa7-40f7-89de-36214730556a'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:18.503695 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-9c1aaa20-dfa7-40f7-89de-36214730556a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8000d42754b4f92be5b1de5201b225f', 'request_id': 'req-9c1aaa20-dfa7-40f7-89de-36214730556a'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:18.520845 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-9c1aaa20-dfa7-40f7-89de-36214730556a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8000d42754b4f92be5b1de5201b225f', 'request_id': 'req-9c1aaa20-dfa7-40f7-89de-36214730556a'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:18.552768 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-9c1aaa20-dfa7-40f7-89de-36214730556a admin admin] The token cache is being invalidate because project e8000d42754b4f92be5b1de5201b225f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:43:18.553070 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-9c1aaa20-dfa7-40f7-89de-36214730556a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:18.555715 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-9c1aaa20-dfa7-40f7-89de-36214730556a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:18.559703 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 832/1653] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:43:18 2022] DELETE /identity/v3/projects/e8000d42754b4f92be5b1de5201b225f => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:18.565754 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc84f06-a3f7-4190-ad4b-e0651211f86d None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:18.566256 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc84f06-a3f7-4190-ad4b-e0651211f86d None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:18.566761 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc84f06-a3f7-4190-ad4b-e0651211f86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:18.574300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1aaa20-dfa7-40f7-89de-36214730556a admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:18.576666 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-9c1aaa20-dfa7-40f7-89de-36214730556a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:18.631137 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-0cc84f06-a3f7-4190-ad4b-e0651211f86d None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:18.637613 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-0cc84f06-a3f7-4190-ad4b-e0651211f86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:18.768725 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 822/1654] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:18.770364 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b42b5e-d234-4e67-8d27-2f9184aec025 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:18.770895 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b42b5e-d234-4e67-8d27-2f9184aec025 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:18.771303 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b42b5e-d234-4e67-8d27-2f9184aec025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:18.810907 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-69b42b5e-d234-4e67-8d27-2f9184aec025 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:18.813179 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-69b42b5e-d234-4e67-8d27-2f9184aec025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:18.817154 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1aaa20-dfa7-40f7-89de-36214730556a admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:18.832372 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d9a328-a7d6-4519-82b6-cc9cddfd8d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:18.833346 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d9a328-a7d6-4519-82b6-cc9cddfd8d58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:18.833631 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d9a328-a7d6-4519-82b6-cc9cddfd8d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:18.833909 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d9a328-a7d6-4519-82b6-cc9cddfd8d58 admin admin] PATH_INFO: `/v3/projects/5393defca80f4e8e8bfbec7a4305d58d` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:18.836375 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d9a328-a7d6-4519-82b6-cc9cddfd8d58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5393defca80f4e8e8bfbec7a4305d58d)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:18.838872 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d9a328-a7d6-4519-82b6-cc9cddfd8d58 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:18.845258 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 823/1655] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:18.850507 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b42b5e-d234-4e67-8d27-2f9184aec025 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:18.871826 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b42b5e-d234-4e67-8d27-2f9184aec025 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:18.879593 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feeda77-8b4e-4387-b28c-72b48247c025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:18.881084 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feeda77-8b4e-4387-b28c-72b48247c025 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:18.881520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feeda77-8b4e-4387-b28c-72b48247c025 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:18.882576 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feeda77-8b4e-4387-b28c-72b48247c025 admin admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:18.884228 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feeda77-8b4e-4387-b28c-72b48247c025 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:43:18.884723 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feeda77-8b4e-4387-b28c-72b48247c025 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:18.886935 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: WARNING py.warnings [None req-9feeda77-8b4e-4387-b28c-72b48247c025 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 17 23:43:18.886935 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: warnings.warn(msg) Aug 17 23:43:18.886935 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: Aug 17 23:43:18.888749 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feeda77-8b4e-4387-b28c-72b48247c025 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:18.913582 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 824/1656] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:43:18.920850 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feeda77-8b4e-4387-b28c-72b48247c025 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:18.945796 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feeda77-8b4e-4387-b28c-72b48247c025 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:18.956210 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b5e19-d3a0-4dc0-99b3-7a4c5ec58bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:18.956829 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b5e19-d3a0-4dc0-99b3-7a4c5ec58bbd admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:18.957105 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b5e19-d3a0-4dc0-99b3-7a4c5ec58bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:18.957400 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b5e19-d3a0-4dc0-99b3-7a4c5ec58bbd admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:18.958101 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788b5e19-d3a0-4dc0-99b3-7a4c5ec58bbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:18.959916 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788b5e19-d3a0-4dc0-99b3-7a4c5ec58bbd admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:18.983805 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-56d9a328-a7d6-4519-82b6-cc9cddfd8d58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5393defca80f4e8e8bfbec7a4305d58d', 'request_id': 'req-56d9a328-a7d6-4519-82b6-cc9cddfd8d58'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:18.995815 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-56d9a328-a7d6-4519-82b6-cc9cddfd8d58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5393defca80f4e8e8bfbec7a4305d58d', 'request_id': 'req-56d9a328-a7d6-4519-82b6-cc9cddfd8d58'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:19.013300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-56d9a328-a7d6-4519-82b6-cc9cddfd8d58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5393defca80f4e8e8bfbec7a4305d58d', 'request_id': 'req-56d9a328-a7d6-4519-82b6-cc9cddfd8d58'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:19.030616 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 825/1657] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:43:18 2022] POST /identity/v3/projects => generated 365 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:19.031733 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b5e19-d3a0-4dc0-99b3-7a4c5ec58bbd admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:19.040296 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:19.047406 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-56d9a328-a7d6-4519-82b6-cc9cddfd8d58 admin admin] The token cache is being invalidate because project 5393defca80f4e8e8bfbec7a4305d58d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:43:19.047903 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-56d9a328-a7d6-4519-82b6-cc9cddfd8d58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:19.050097 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-56d9a328-a7d6-4519-82b6-cc9cddfd8d58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:19.055028 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 833/1658] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:43:18 2022] DELETE /identity/v3/projects/5393defca80f4e8e8bfbec7a4305d58d => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:19.061837 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727cd3d-c85d-4471-8edb-2c2c49086c07 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:19.062130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727cd3d-c85d-4471-8edb-2c2c49086c07 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:19.062493 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727cd3d-c85d-4471-8edb-2c2c49086c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:19.116746 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-b727cd3d-c85d-4471-8edb-2c2c49086c07 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:19.119424 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-b727cd3d-c85d-4471-8edb-2c2c49086c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:19.214925 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b5e19-d3a0-4dc0-99b3-7a4c5ec58bbd admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:19.234825 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef63f95-06a8-4470-b128-e92bbe2a21ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:19.235466 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef63f95-06a8-4470-b128-e92bbe2a21ad admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:19.235745 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef63f95-06a8-4470-b128-e92bbe2a21ad admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:19.236046 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef63f95-06a8-4470-b128-e92bbe2a21ad admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:19.236683 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef63f95-06a8-4470-b128-e92bbe2a21ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:19.238639 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef63f95-06a8-4470-b128-e92bbe2a21ad admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:19.248916 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 834/1659] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:19.258332 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415c174-ad30-4025-82b5-9bfc6e76ebd2 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:19.258626 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415c174-ad30-4025-82b5-9bfc6e76ebd2 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:19.258903 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415c174-ad30-4025-82b5-9bfc6e76ebd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:19.261710 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:19.334346 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 826/1660] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:43:19 2022] POST /identity/v3/users => generated 283 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:19.341558 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef63f95-06a8-4470-b128-e92bbe2a21ad admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:19.346125 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-8ef63f95-06a8-4470-b128-e92bbe2a21ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:19.357674 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-3415c174-ad30-4025-82b5-9bfc6e76ebd2 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:19.393496 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-3415c174-ad30-4025-82b5-9bfc6e76ebd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:19.408188 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef63f95-06a8-4470-b128-e92bbe2a21ad admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:19.421735 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a82ce7-4170-4071-a975-dd3265a694d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:19.422899 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a82ce7-4170-4071-a975-dd3265a694d9 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:19.423183 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a82ce7-4170-4071-a975-dd3265a694d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:19.423457 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a82ce7-4170-4071-a975-dd3265a694d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:19.424206 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a82ce7-4170-4071-a975-dd3265a694d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:19.425871 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a82ce7-4170-4071-a975-dd3265a694d9 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:19.426904 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 835/1661] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:19.431091 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415c174-ad30-4025-82b5-9bfc6e76ebd2 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:19.455687 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415c174-ad30-4025-82b5-9bfc6e76ebd2 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:19.456944 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 827/1662] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:19.460917 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a82ce7-4170-4071-a975-dd3265a694d9 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:19.462621 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6a4c16-5ef9-4d3e-bfd4-5346aa904031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:19.463433 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a4c16-5ef9-4d3e-bfd4-5346aa904031 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:19.464040 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a4c16-5ef9-4d3e-bfd4-5346aa904031 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:19.466244 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a4c16-5ef9-4d3e-bfd4-5346aa904031 admin admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:19.467054 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6a4c16-5ef9-4d3e-bfd4-5346aa904031 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:43:19.467473 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6a4c16-5ef9-4d3e-bfd4-5346aa904031 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:19.473690 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6a4c16-5ef9-4d3e-bfd4-5346aa904031 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:19.479844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a82ce7-4170-4071-a975-dd3265a694d9 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:19.498639 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f623b9-23b0-4165-9494-6e648d90a443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:19.499185 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f623b9-23b0-4165-9494-6e648d90a443 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:19.499463 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f623b9-23b0-4165-9494-6e648d90a443 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:19.499733 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f623b9-23b0-4165-9494-6e648d90a443 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:19.500419 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f623b9-23b0-4165-9494-6e648d90a443 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:19.502090 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f623b9-23b0-4165-9494-6e648d90a443 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:19.504063 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 836/1663] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:43:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:43:19.509548 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6a4c16-5ef9-4d3e-bfd4-5346aa904031 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:19.543472 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6a4c16-5ef9-4d3e-bfd4-5346aa904031 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:19.544155 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 828/1664] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:19.547889 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3a860-c395-4c4b-acf8-5a61ffa23acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:19.548674 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3a860-c395-4c4b-acf8-5a61ffa23acc admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:19.549185 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3a860-c395-4c4b-acf8-5a61ffa23acc admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:19.551951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f623b9-23b0-4165-9494-6e648d90a443 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:19.552426 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3a860-c395-4c4b-acf8-5a61ffa23acc admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:19.553314 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f3a860-c395-4c4b-acf8-5a61ffa23acc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:19.555382 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f3a860-c395-4c4b-acf8-5a61ffa23acc admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:19.589782 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f623b9-23b0-4165-9494-6e648d90a443 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:19.607079 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8ec141-a0f2-45c6-9359-bc9cd6b45f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:19.607762 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ec141-a0f2-45c6-9359-bc9cd6b45f41 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:19.608078 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ec141-a0f2-45c6-9359-bc9cd6b45f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:19.608342 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ec141-a0f2-45c6-9359-bc9cd6b45f41 admin admin] PATH_INFO: `/v3/projects/ee9232ccb9f74a76a28ca7c758375f1b/users/e5e31de870fd4def90df97abc9714d38/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:19.622038 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 837/1665] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:43:19 2022] POST /identity/v3/projects => generated 405 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:19.627836 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3a860-c395-4c4b-acf8-5a61ffa23acc admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:19.660622 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8ec141-a0f2-45c6-9359-bc9cd6b45f41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee9232ccb9f74a76a28ca7c758375f1b, user_id=e5e31de870fd4def90df97abc9714d38, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:19.669641 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8ec141-a0f2-45c6-9359-bc9cd6b45f41 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:19.696758 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 829/1666] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:43:19 2022] PUT /identity/v3/projects/ee9232ccb9f74a76a28ca7c758375f1b/users/e5e31de870fd4def90df97abc9714d38/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:19.707468 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc3e44-8f6a-455a-802a-a005462661f4 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:19.708300 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc3e44-8f6a-455a-802a-a005462661f4 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:19.708840 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc3e44-8f6a-455a-802a-a005462661f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:19.777138 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-a0bc3e44-8f6a-455a-802a-a005462661f4 None None] MFA Rules not processed for user `e5e31de870fd4def90df97abc9714d38`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:19.778653 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-a0bc3e44-8f6a-455a-802a-a005462661f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:19.783464 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3a860-c395-4c4b-acf8-5a61ffa23acc admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:19.908011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d7c178-1c8d-49d6-b3f5-3090ba819b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:19.908827 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7c178-1c8d-49d6-b3f5-3090ba819b31 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:19.909227 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7c178-1c8d-49d6-b3f5-3090ba819b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:19.909879 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7c178-1c8d-49d6-b3f5-3090ba819b31 admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:19.910655 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d7c178-1c8d-49d6-b3f5-3090ba819b31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:19.912765 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d7c178-1c8d-49d6-b3f5-3090ba819b31 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:19.921533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 830/1667] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:19.923133 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bc3e44-8f6a-455a-802a-a005462661f4 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:19.937413 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bc3e44-8f6a-455a-802a-a005462661f4 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:19.938629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:19.944344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a019cdf-0ba8-4e96-99ef-8947fb9e67bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:19.945133 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a019cdf-0ba8-4e96-99ef-8947fb9e67bf admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:19.945737 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a019cdf-0ba8-4e96-99ef-8947fb9e67bf admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:19.946134 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a019cdf-0ba8-4e96-99ef-8947fb9e67bf admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:19.946988 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a019cdf-0ba8-4e96-99ef-8947fb9e67bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:19.950012 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a019cdf-0ba8-4e96-99ef-8947fb9e67bf admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:19.958998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 838/1668] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:43:19 2022] POST /identity/v3/users => generated 303 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:19.963814 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d7c178-1c8d-49d6-b3f5-3090ba819b31 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:19.984560 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d7c178-1c8d-49d6-b3f5-3090ba819b31 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:19.999006 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 831/1669] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:43:19 2022] POST /identity/v3/projects => generated 365 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:20.000346 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a019cdf-0ba8-4e96-99ef-8947fb9e67bf admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:20.001976 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a623741f-2558-46ba-9e11-b9e96be32ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:20.002725 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623741f-2558-46ba-9e11-b9e96be32ff8 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:20.003118 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623741f-2558-46ba-9e11-b9e96be32ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:20.003552 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623741f-2558-46ba-9e11-b9e96be32ff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:20.004304 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a623741f-2558-46ba-9e11-b9e96be32ff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:20.006160 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a623741f-2558-46ba-9e11-b9e96be32ff8 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:20.033317 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 839/1670] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:20.036260 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a623741f-2558-46ba-9e11-b9e96be32ff8 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:20.058738 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a623741f-2558-46ba-9e11-b9e96be32ff8 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:20.148034 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a019cdf-0ba8-4e96-99ef-8947fb9e67bf admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:20.159358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70e2546-b02a-40d8-8eb4-1a11a6919f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:20.160376 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e2546-b02a-40d8-8eb4-1a11a6919f58 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:20.161848 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e2546-b02a-40d8-8eb4-1a11a6919f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:20.162120 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e2546-b02a-40d8-8eb4-1a11a6919f58 admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:20.164875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70e2546-b02a-40d8-8eb4-1a11a6919f58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:20.170103 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70e2546-b02a-40d8-8eb4-1a11a6919f58 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:20.177278 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb94ebc-fdec-46f5-bbc9-59c8d2c3d252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:20.177910 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb94ebc-fdec-46f5-bbc9-59c8d2c3d252 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:20.178189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb94ebc-fdec-46f5-bbc9-59c8d2c3d252 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:20.178467 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb94ebc-fdec-46f5-bbc9-59c8d2c3d252 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:20.186537 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb94ebc-fdec-46f5-bbc9-59c8d2c3d252 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:20.189738 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb94ebc-fdec-46f5-bbc9-59c8d2c3d252 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:20.219405 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 840/1671] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:20.223060 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:20.223195 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb94ebc-fdec-46f5-bbc9-59c8d2c3d252 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:20.233470 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 832/1672] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:43:19 2022] POST /identity/v3/users => generated 289 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:20.236255 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70e2546-b02a-40d8-8eb4-1a11a6919f58 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:20.259408 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb94ebc-fdec-46f5-bbc9-59c8d2c3d252 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:20.264888 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70e2546-b02a-40d8-8eb4-1a11a6919f58 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:20.270865 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00478982-8c09-41ac-9c10-d3ba66ee2fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:20.271611 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00478982-8c09-41ac-9c10-d3ba66ee2fda admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:20.271899 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00478982-8c09-41ac-9c10-d3ba66ee2fda admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:20.272338 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00478982-8c09-41ac-9c10-d3ba66ee2fda admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:20.273000 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00478982-8c09-41ac-9c10-d3ba66ee2fda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:20.274639 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00478982-8c09-41ac-9c10-d3ba66ee2fda admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:20.295076 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04455dd3-f6bd-4020-a012-e7431b287bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:20.295766 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04455dd3-f6bd-4020-a012-e7431b287bff admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:20.296125 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04455dd3-f6bd-4020-a012-e7431b287bff admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:20.296402 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04455dd3-f6bd-4020-a012-e7431b287bff admin admin] PATH_INFO: `/v3/projects/65bfd9750d3f46e7bf7e6c43741d0a6b/users/988dcf6365944d6f9a0906f64fcec51e/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:20.302398 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 833/1673] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:20.306626 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00478982-8c09-41ac-9c10-d3ba66ee2fda admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:20.334656 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00478982-8c09-41ac-9c10-d3ba66ee2fda admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:20.346059 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db26bd-8012-4b36-a321-80f4804d160d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:20.346971 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db26bd-8012-4b36-a321-80f4804d160d admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:20.347395 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db26bd-8012-4b36-a321-80f4804d160d admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:20.347925 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db26bd-8012-4b36-a321-80f4804d160d admin admin] PATH_INFO: `/v3/projects/be508c6c7750485ea29d0d6bc41b22c2/users/2a8a021cda844ce1bdbf67510d8345f3/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:20.361257 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04455dd3-f6bd-4020-a012-e7431b287bff admin admin] RBAC: Authorizing `identity:create_grant(project_id=65bfd9750d3f46e7bf7e6c43741d0a6b, user_id=988dcf6365944d6f9a0906f64fcec51e, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:20.365270 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04455dd3-f6bd-4020-a012-e7431b287bff admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:20.396811 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db26bd-8012-4b36-a321-80f4804d160d admin admin] RBAC: Authorizing `identity:create_grant(project_id=be508c6c7750485ea29d0d6bc41b22c2, user_id=2a8a021cda844ce1bdbf67510d8345f3, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:20.402278 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 841/1674] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:43:20 2022] PUT /identity/v3/projects/65bfd9750d3f46e7bf7e6c43741d0a6b/users/988dcf6365944d6f9a0906f64fcec51e/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:43:20.404243 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db26bd-8012-4b36-a321-80f4804d160d admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:20.413777 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4159d-af08-46ff-a981-06ac6c3615da None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:20.414085 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4159d-af08-46ff-a981-06ac6c3615da None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:20.414419 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4159d-af08-46ff-a981-06ac6c3615da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:20.423769 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 834/1675] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:43:20 2022] PUT /identity/v3/projects/be508c6c7750485ea29d0d6bc41b22c2/users/2a8a021cda844ce1bdbf67510d8345f3/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:20.427656 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db26bd-8012-4b36-a321-80f4804d160d admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:20.467435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-a2e4159d-af08-46ff-a981-06ac6c3615da None None] MFA Rules not processed for user `988dcf6365944d6f9a0906f64fcec51e`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:20.468820 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-a2e4159d-af08-46ff-a981-06ac6c3615da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:20.530753 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db26bd-8012-4b36-a321-80f4804d160d admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:20.539913 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e23830-49e5-468f-b557-687d17e8cdf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:20.548578 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e23830-49e5-468f-b557-687d17e8cdf8 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:20.549029 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e23830-49e5-468f-b557-687d17e8cdf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:20.549453 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e23830-49e5-468f-b557-687d17e8cdf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:20.550283 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e23830-49e5-468f-b557-687d17e8cdf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:20.553811 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e23830-49e5-468f-b557-687d17e8cdf8 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:20.569831 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 835/1676] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:20.574176 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e23830-49e5-468f-b557-687d17e8cdf8 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:20.598001 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 842/1677] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:20 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:20.599416 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e4159d-af08-46ff-a981-06ac6c3615da None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:20.602366 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e23830-49e5-468f-b557-687d17e8cdf8 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:20.609310 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f505fb3-ec5f-4740-a2c6-dc0da5b07394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:20.610134 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f505fb3-ec5f-4740-a2c6-dc0da5b07394 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:20.610529 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f505fb3-ec5f-4740-a2c6-dc0da5b07394 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:20.610916 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f505fb3-ec5f-4740-a2c6-dc0da5b07394 admin admin] PATH_INFO: `/v3/projects/be508c6c7750485ea29d0d6bc41b22c2/users/2a8a021cda844ce1bdbf67510d8345f3/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:20.614383 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f505fb3-ec5f-4740-a2c6-dc0da5b07394 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be508c6c7750485ea29d0d6bc41b22c2, user_id=2a8a021cda844ce1bdbf67510d8345f3, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:20.617929 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f505fb3-ec5f-4740-a2c6-dc0da5b07394 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:20.626492 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e4159d-af08-46ff-a981-06ac6c3615da None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:20.655814 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 836/1678] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:43:20 2022] PUT /identity/v3/projects/be508c6c7750485ea29d0d6bc41b22c2/users/2a8a021cda844ce1bdbf67510d8345f3/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:43:20.671410 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e803570-f958-4dcc-ae6d-77e2d14caac2 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:20.674037 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e803570-f958-4dcc-ae6d-77e2d14caac2 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:20.674483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e803570-f958-4dcc-ae6d-77e2d14caac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:20.728601 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:20.763388 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b201197-dcd8-4e84-8de8-43301578a7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:20.764056 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b201197-dcd8-4e84-8de8-43301578a7eb admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:20.764303 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b201197-dcd8-4e84-8de8-43301578a7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:20.764571 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b201197-dcd8-4e84-8de8-43301578a7eb admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:20.765200 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b201197-dcd8-4e84-8de8-43301578a7eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:20.771543 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b201197-dcd8-4e84-8de8-43301578a7eb admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:20.781221 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-7e803570-f958-4dcc-ae6d-77e2d14caac2 None None] MFA Rules not processed for user `2a8a021cda844ce1bdbf67510d8345f3`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:20.782820 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-7e803570-f958-4dcc-ae6d-77e2d14caac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:20.838128 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 843/1679] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:43:20 2022] POST /identity/v3/projects => generated 403 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:20.838635 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b201197-dcd8-4e84-8de8-43301578a7eb admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:20.928471 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 837/1680] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:20 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:20.940705 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e803570-f958-4dcc-ae6d-77e2d14caac2 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:20.943240 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-7e803570-f958-4dcc-ae6d-77e2d14caac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:20.975324 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b201197-dcd8-4e84-8de8-43301578a7eb admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:20.982071 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec1eba-406c-4e8e-a434-776a3eddc938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:20.982506 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec1eba-406c-4e8e-a434-776a3eddc938 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:20.982799 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec1eba-406c-4e8e-a434-776a3eddc938 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:20.983080 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec1eba-406c-4e8e-a434-776a3eddc938 admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:20.983734 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ec1eba-406c-4e8e-a434-776a3eddc938 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:20.985983 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ec1eba-406c-4e8e-a434-776a3eddc938 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:21.014679 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:21.024350 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 844/1681] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:43:20 2022] POST /identity/v3/users => generated 308 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:21.025194 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec1eba-406c-4e8e-a434-776a3eddc938 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:21.030092 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-46ec1eba-406c-4e8e-a434-776a3eddc938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:21.054095 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e803570-f958-4dcc-ae6d-77e2d14caac2 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:21.092981 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec1eba-406c-4e8e-a434-776a3eddc938 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:21.113423 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e77ef-2640-4e35-89b1-27a35675e37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:21.113955 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e77ef-2640-4e35-89b1-27a35675e37a admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:21.114241 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e77ef-2640-4e35-89b1-27a35675e37a admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:21.114521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e77ef-2640-4e35-89b1-27a35675e37a admin admin] PATH_INFO: `/v3/users/bfc475462cf04deb8b1a1c96665d9ea7` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:21.120787 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e77ef-2640-4e35-89b1-27a35675e37a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfc475462cf04deb8b1a1c96665d9ea7)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:21.126662 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e77ef-2640-4e35-89b1-27a35675e37a admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:21.145255 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f66a2a-def1-4986-b593-204a0e96ad5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:21.145960 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f66a2a-def1-4986-b593-204a0e96ad5a service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:21.147566 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f66a2a-def1-4986-b593-204a0e96ad5a service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:21.147680 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f66a2a-def1-4986-b593-204a0e96ad5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:21.233718 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-b18e77ef-2640-4e35-89b1-27a35675e37a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfc475462cf04deb8b1a1c96665d9ea7'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:21.234957 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-b18e77ef-2640-4e35-89b1-27a35675e37a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfc475462cf04deb8b1a1c96665d9ea7'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:21.237701 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-b18e77ef-2640-4e35-89b1-27a35675e37a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfc475462cf04deb8b1a1c96665d9ea7'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:21.256484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-b18e77ef-2640-4e35-89b1-27a35675e37a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfc475462cf04deb8b1a1c96665d9ea7'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:21.265040 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-b18e77ef-2640-4e35-89b1-27a35675e37a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfc475462cf04deb8b1a1c96665d9ea7'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:21.270621 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f66a2a-def1-4986-b593-204a0e96ad5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:21.272571 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f66a2a-def1-4986-b593-204a0e96ad5a service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:21.286916 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 838/1682] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:20 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:21.287910 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f66a2a-def1-4986-b593-204a0e96ad5a service cinder] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:21.301274 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f66a2a-def1-4986-b593-204a0e96ad5a service cinder] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:21.306629 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e8068-e86c-4a5d-8e43-1c4da358d9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:21.308041 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e8068-e86c-4a5d-8e43-1c4da358d9ee admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:21.320501 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e8068-e86c-4a5d-8e43-1c4da358d9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:21.321572 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e8068-e86c-4a5d-8e43-1c4da358d9ee admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:21.323161 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105e8068-e86c-4a5d-8e43-1c4da358d9ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:21.328569 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105e8068-e86c-4a5d-8e43-1c4da358d9ee admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:21.331728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 845/1683] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:43:21 2022] DELETE /identity/v3/users/bfc475462cf04deb8b1a1c96665d9ea7 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:21.335072 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeff474-a4ea-4f07-a1a8-cc0ae5ef9648 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:21.335479 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeff474-a4ea-4f07-a1a8-cc0ae5ef9648 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:21.335762 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeff474-a4ea-4f07-a1a8-cc0ae5ef9648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:21.372429 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 839/1684] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:21.377155 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e8068-e86c-4a5d-8e43-1c4da358d9ee admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:21.381928 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-105e8068-e86c-4a5d-8e43-1c4da358d9ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:21.388085 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-cdeff474-a4ea-4f07-a1a8-cc0ae5ef9648 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:21.390674 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-cdeff474-a4ea-4f07-a1a8-cc0ae5ef9648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:21.415750 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e8068-e86c-4a5d-8e43-1c4da358d9ee admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:21.465978 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:21.545646 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 846/1685] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:21.548474 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeff474-a4ea-4f07-a1a8-cc0ae5ef9648 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:21.550294 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-cdeff474-a4ea-4f07-a1a8-cc0ae5ef9648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:21.573079 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8aed95-8e39-41d9-bc63-de60e1471f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:21.574259 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8aed95-8e39-41d9-bc63-de60e1471f71 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:21.574259 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8aed95-8e39-41d9-bc63-de60e1471f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:21.574259 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8aed95-8e39-41d9-bc63-de60e1471f71 admin admin] PATH_INFO: `/v3/projects/59870531b2514c26b0bb43bf5cfc0316/users/ad68253397bc47c986e21b4ac899d6ab/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:21.630349 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8aed95-8e39-41d9-bc63-de60e1471f71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59870531b2514c26b0bb43bf5cfc0316, user_id=ad68253397bc47c986e21b4ac899d6ab, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:21.633107 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8aed95-8e39-41d9-bc63-de60e1471f71 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:21.658220 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 840/1686] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:43:21 2022] PUT /identity/v3/projects/59870531b2514c26b0bb43bf5cfc0316/users/ad68253397bc47c986e21b4ac899d6ab/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:21.659171 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8aed95-8e39-41d9-bc63-de60e1471f71 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:21.665275 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-0e8aed95-8e39-41d9-bc63-de60e1471f71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:21.669038 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeff474-a4ea-4f07-a1a8-cc0ae5ef9648 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:21.698336 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:21.811100 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29da98e-0e83-4e10-9050-00e2761d3880 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:21.811788 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29da98e-0e83-4e10-9050-00e2761d3880 service cinder] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:21.812108 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29da98e-0e83-4e10-9050-00e2761d3880 service cinder] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:21.819271 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29da98e-0e83-4e10-9050-00e2761d3880 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:21.824153 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-e29da98e-0e83-4e10-9050-00e2761d3880 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:21.872294 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8aed95-8e39-41d9-bc63-de60e1471f71 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:21.882821 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c7362-4550-48f7-91ff-c30c9865caba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:21.883506 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c7362-4550-48f7-91ff-c30c9865caba service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:21.883793 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c7362-4550-48f7-91ff-c30c9865caba service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:21.884096 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c7362-4550-48f7-91ff-c30c9865caba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:22.013810 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c7362-4550-48f7-91ff-c30c9865caba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:22.015602 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c7362-4550-48f7-91ff-c30c9865caba service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:22.035067 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 841/1687] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:22.037604 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29da98e-0e83-4e10-9050-00e2761d3880 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:22.037836 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c7362-4550-48f7-91ff-c30c9865caba service neutron] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:22.039141 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29da98e-0e83-4e10-9050-00e2761d3880 service cinder] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:22.061958 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 847/1688] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:22.064170 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c7362-4550-48f7-91ff-c30c9865caba service neutron] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:22.088350 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ad7465-4380-4bbb-a9ba-d9271539b82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:22.088996 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad7465-4380-4bbb-a9ba-d9271539b82a admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:22.089281 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad7465-4380-4bbb-a9ba-d9271539b82a admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:22.089559 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad7465-4380-4bbb-a9ba-d9271539b82a admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:22.105535 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ad7465-4380-4bbb-a9ba-d9271539b82a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:22.109202 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ad7465-4380-4bbb-a9ba-d9271539b82a admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:22.149760 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 842/1689] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:22.152402 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29da98e-0e83-4e10-9050-00e2761d3880 service cinder] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:22.170379 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29da98e-0e83-4e10-9050-00e2761d3880 service cinder] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:22.182928 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5913dde1-bc14-47cc-b467-0204b4820f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:22.184014 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913dde1-bc14-47cc-b467-0204b4820f4e admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:22.186253 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913dde1-bc14-47cc-b467-0204b4820f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:22.186604 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913dde1-bc14-47cc-b467-0204b4820f4e admin admin] PATH_INFO: `/v3/projects/59870531b2514c26b0bb43bf5cfc0316/users/ad68253397bc47c986e21b4ac899d6ab/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:22.194160 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5913dde1-bc14-47cc-b467-0204b4820f4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=59870531b2514c26b0bb43bf5cfc0316, user_id=ad68253397bc47c986e21b4ac899d6ab, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:22.199621 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5913dde1-bc14-47cc-b467-0204b4820f4e admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:22.239348 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 848/1690] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:43:22 2022] PUT /identity/v3/projects/59870531b2514c26b0bb43bf5cfc0316/users/ad68253397bc47c986e21b4ac899d6ab/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:22.241199 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2113e95-4f93-4e17-86b3-639945cee6c5 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:22.241485 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2113e95-4f93-4e17-86b3-639945cee6c5 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:22.241932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2113e95-4f93-4e17-86b3-639945cee6c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:22.319418 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-c2113e95-4f93-4e17-86b3-639945cee6c5 None None] MFA Rules not processed for user `ad68253397bc47c986e21b4ac899d6ab`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:22.320951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-c2113e95-4f93-4e17-86b3-639945cee6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:22.457699 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 843/1691] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:22 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:22.470288 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5913dde1-bc14-47cc-b467-0204b4820f4e admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:22.476342 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-5913dde1-bc14-47cc-b467-0204b4820f4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:22.480484 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd40a40-01a7-4a65-b164-51986e64cef6 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:22.480780 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd40a40-01a7-4a65-b164-51986e64cef6 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:22.481058 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd40a40-01a7-4a65-b164-51986e64cef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:22.539031 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-4dd40a40-01a7-4a65-b164-51986e64cef6 None None] MFA Rules not processed for user `21e06d9bb5a240459ef3298a990a42f2`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:22.540369 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-4dd40a40-01a7-4a65-b164-51986e64cef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:22.555878 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:22.599828 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:22.634105 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5913dde1-bc14-47cc-b467-0204b4820f4e admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:22.645985 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848403d1-38dc-4c9f-8f0e-836d385e0e12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:22.647066 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848403d1-38dc-4c9f-8f0e-836d385e0e12 service cinder] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:22.647178 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848403d1-38dc-4c9f-8f0e-836d385e0e12 service cinder] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:22.647532 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848403d1-38dc-4c9f-8f0e-836d385e0e12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:22.666674 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848403d1-38dc-4c9f-8f0e-836d385e0e12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:22.668297 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848403d1-38dc-4c9f-8f0e-836d385e0e12 service cinder] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:22.687634 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 849/1692] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:22 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:22.723368 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 844/1693] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:43:22.732607 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848403d1-38dc-4c9f-8f0e-836d385e0e12 service cinder] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:22.734930 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-848403d1-38dc-4c9f-8f0e-836d385e0e12 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:22.934629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848403d1-38dc-4c9f-8f0e-836d385e0e12 service cinder] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:22.944958 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447b827-f55e-409b-b4e4-a14dcee7f7ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:22.947021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447b827-f55e-409b-b4e4-a14dcee7f7ea service nova] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:22.948088 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447b827-f55e-409b-b4e4-a14dcee7f7ea service nova] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:22.948528 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447b827-f55e-409b-b4e4-a14dcee7f7ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:22.961494 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d447b827-f55e-409b-b4e4-a14dcee7f7ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:22.964387 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d447b827-f55e-409b-b4e4-a14dcee7f7ea service nova] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:22.975777 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd40a40-01a7-4a65-b164-51986e64cef6 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:23.015398 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 850/1694] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:22 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:23.142964 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd40a40-01a7-4a65-b164-51986e64cef6 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:23.158240 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3676d5-5b23-4d50-ae0c-2f3a06caefcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:23.158896 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3676d5-5b23-4d50-ae0c-2f3a06caefcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:23.159171 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3676d5-5b23-4d50-ae0c-2f3a06caefcc admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:23.159708 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3676d5-5b23-4d50-ae0c-2f3a06caefcc admin admin] PATH_INFO: `/v3/users/988dcf6365944d6f9a0906f64fcec51e` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:23.164112 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3676d5-5b23-4d50-ae0c-2f3a06caefcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=988dcf6365944d6f9a0906f64fcec51e)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:23.166389 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3676d5-5b23-4d50-ae0c-2f3a06caefcc admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:23.270431 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-cc3676d5-5b23-4d50-ae0c-2f3a06caefcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '988dcf6365944d6f9a0906f64fcec51e'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:23.272657 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-cc3676d5-5b23-4d50-ae0c-2f3a06caefcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '988dcf6365944d6f9a0906f64fcec51e'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:23.310509 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-cc3676d5-5b23-4d50-ae0c-2f3a06caefcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '988dcf6365944d6f9a0906f64fcec51e'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:23.327996 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-cc3676d5-5b23-4d50-ae0c-2f3a06caefcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '988dcf6365944d6f9a0906f64fcec51e'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:23.329106 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-cc3676d5-5b23-4d50-ae0c-2f3a06caefcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '988dcf6365944d6f9a0906f64fcec51e'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:23.357509 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 845/1695] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:43:22 2022] DELETE /identity/v3/users/988dcf6365944d6f9a0906f64fcec51e => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:23.362085 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c1612-72c4-47e8-96f4-a4ec1a068297 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:23.362430 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c1612-72c4-47e8-96f4-a4ec1a068297 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:23.362719 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c1612-72c4-47e8-96f4-a4ec1a068297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:23.405592 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-153c1612-72c4-47e8-96f4-a4ec1a068297 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:23.406863 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-153c1612-72c4-47e8-96f4-a4ec1a068297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:23.585045 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 851/1696] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:23.861883 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3676d5-5b23-4d50-ae0c-2f3a06caefcc admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:23.863979 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-cc3676d5-5b23-4d50-ae0c-2f3a06caefcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:24.041902 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c1612-72c4-47e8-96f4-a4ec1a068297 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:24.043364 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-153c1612-72c4-47e8-96f4-a4ec1a068297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:24.046049 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:24.078866 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c1612-72c4-47e8-96f4-a4ec1a068297 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:24.082030 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3676d5-5b23-4d50-ae0c-2f3a06caefcc admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:24.097909 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90535d28-6c51-473a-8376-96e1a3a8a9d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:24.099525 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90535d28-6c51-473a-8376-96e1a3a8a9d4 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:24.107954 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e38f0e3-f7c9-4f10-8227-017d29561b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:24.108659 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38f0e3-f7c9-4f10-8227-017d29561b5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:24.109022 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38f0e3-f7c9-4f10-8227-017d29561b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:24.109180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90535d28-6c51-473a-8376-96e1a3a8a9d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:24.109282 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38f0e3-f7c9-4f10-8227-017d29561b5e admin admin] PATH_INFO: `/v3/projects/8069466b28f443ea8ab87ab598c1ede2` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:24.109671 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90535d28-6c51-473a-8376-96e1a3a8a9d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:24.113213 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e38f0e3-f7c9-4f10-8227-017d29561b5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8069466b28f443ea8ab87ab598c1ede2)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:24.114963 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e38f0e3-f7c9-4f10-8227-017d29561b5e admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:24.210894 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90535d28-6c51-473a-8376-96e1a3a8a9d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:24.218138 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90535d28-6c51-473a-8376-96e1a3a8a9d4 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:24.291541 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 846/1697] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:24.378486 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-1e38f0e3-f7c9-4f10-8227-017d29561b5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8069466b28f443ea8ab87ab598c1ede2', 'request_id': 'req-1e38f0e3-f7c9-4f10-8227-017d29561b5e'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:24.410628 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:24.425778 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-1e38f0e3-f7c9-4f10-8227-017d29561b5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8069466b28f443ea8ab87ab598c1ede2', 'request_id': 'req-1e38f0e3-f7c9-4f10-8227-017d29561b5e'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:24.465658 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-1e38f0e3-f7c9-4f10-8227-017d29561b5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8069466b28f443ea8ab87ab598c1ede2', 'request_id': 'req-1e38f0e3-f7c9-4f10-8227-017d29561b5e'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:24.510144 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-1e38f0e3-f7c9-4f10-8227-017d29561b5e admin admin] The token cache is being invalidate because project 8069466b28f443ea8ab87ab598c1ede2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:43:24.513090 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-1e38f0e3-f7c9-4f10-8227-017d29561b5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:24.514329 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-1e38f0e3-f7c9-4f10-8227-017d29561b5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:24.517399 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 852/1698] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:43:24 2022] DELETE /identity/v3/projects/8069466b28f443ea8ab87ab598c1ede2 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:43:24.523004 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90535d28-6c51-473a-8376-96e1a3a8a9d4 service neutron] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:24.570351 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90535d28-6c51-473a-8376-96e1a3a8a9d4 service neutron] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:24.826714 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c95f149-c712-45a1-a46b-1e2ee844b215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:24.826955 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95f149-c712-45a1-a46b-1e2ee844b215 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:24.827201 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95f149-c712-45a1-a46b-1e2ee844b215 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:24.827478 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95f149-c712-45a1-a46b-1e2ee844b215 admin admin] PATH_INFO: `/v3/users/4098d610971e462ab2c9e6d80b85355c` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:24.837258 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c95f149-c712-45a1-a46b-1e2ee844b215 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4098d610971e462ab2c9e6d80b85355c)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:24.839825 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c95f149-c712-45a1-a46b-1e2ee844b215 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:24.967245 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-8c95f149-c712-45a1-a46b-1e2ee844b215 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4098d610971e462ab2c9e6d80b85355c'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:24.971762 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-8c95f149-c712-45a1-a46b-1e2ee844b215 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4098d610971e462ab2c9e6d80b85355c'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:25.015336 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-8c95f149-c712-45a1-a46b-1e2ee844b215 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4098d610971e462ab2c9e6d80b85355c'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:25.043156 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-8c95f149-c712-45a1-a46b-1e2ee844b215 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4098d610971e462ab2c9e6d80b85355c'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:25.043794 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-8c95f149-c712-45a1-a46b-1e2ee844b215 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4098d610971e462ab2c9e6d80b85355c'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:25.075824 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 847/1699] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:43:24 2022] DELETE /identity/v3/users/4098d610971e462ab2c9e6d80b85355c => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:25.401633 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e38f0e3-f7c9-4f10-8227-017d29561b5e admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:25.403618 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-1e38f0e3-f7c9-4f10-8227-017d29561b5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:25.436444 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e38f0e3-f7c9-4f10-8227-017d29561b5e admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:25.494659 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0499b0b-0d89-4512-bc6b-555c67be01a2 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:25.495206 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0499b0b-0d89-4512-bc6b-555c67be01a2 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:25.495662 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0499b0b-0d89-4512-bc6b-555c67be01a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:25.552049 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-f0499b0b-0d89-4512-bc6b-555c67be01a2 None None] MFA Rules not processed for user `c67dec924b7e4f0aa0874e6e21879889`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:25.553302 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-f0499b0b-0d89-4512-bc6b-555c67be01a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:25.607780 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c696b5c-aca3-491c-956f-e2950d0d333b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:25.608464 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c696b5c-aca3-491c-956f-e2950d0d333b admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:25.608739 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c696b5c-aca3-491c-956f-e2950d0d333b admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:25.609008 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c696b5c-aca3-491c-956f-e2950d0d333b admin admin] PATH_INFO: `/v3/users/140b34e8949846549815df130d6ab172` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:25.613992 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c696b5c-aca3-491c-956f-e2950d0d333b admin admin] RBAC: Authorizing `identity:delete_user(user_id=140b34e8949846549815df130d6ab172)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:25.636004 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c696b5c-aca3-491c-956f-e2950d0d333b admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:25.750308 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:25.805030 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 848/1700] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:25 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:25.806414 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac91a5-a3f7-4472-b5c8-5e9dc181066a None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:25.806763 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac91a5-a3f7-4472-b5c8-5e9dc181066a None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:25.807036 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac91a5-a3f7-4472-b5c8-5e9dc181066a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:25.811294 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-2c696b5c-aca3-491c-956f-e2950d0d333b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '140b34e8949846549815df130d6ab172'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:25.813961 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-2c696b5c-aca3-491c-956f-e2950d0d333b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '140b34e8949846549815df130d6ab172'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:25.816088 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-2c696b5c-aca3-491c-956f-e2950d0d333b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '140b34e8949846549815df130d6ab172'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:25.856355 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-2c696b5c-aca3-491c-956f-e2950d0d333b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '140b34e8949846549815df130d6ab172'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:25.867453 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-2c696b5c-aca3-491c-956f-e2950d0d333b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '140b34e8949846549815df130d6ab172'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:25.882234 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-9fac91a5-a3f7-4472-b5c8-5e9dc181066a None None] MFA Rules not processed for user `c67dec924b7e4f0aa0874e6e21879889`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:25.884800 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-9fac91a5-a3f7-4472-b5c8-5e9dc181066a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:25.907791 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 849/1701] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:25 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:25.909087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 853/1702] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:43:25 2022] DELETE /identity/v3/users/140b34e8949846549815df130d6ab172 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:25.909273 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fac91a5-a3f7-4472-b5c8-5e9dc181066a None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:25.912011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c696b5c-aca3-491c-956f-e2950d0d333b admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:25.912968 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-9fac91a5-a3f7-4472-b5c8-5e9dc181066a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:25.916661 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-2c696b5c-aca3-491c-956f-e2950d0d333b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:25.943007 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fac91a5-a3f7-4472-b5c8-5e9dc181066a None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:26.046719 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:26.063191 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20fcacc-4504-46bb-bcfa-db17e680bae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:26.064553 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fcacc-4504-46bb-bcfa-db17e680bae6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:26.064553 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fcacc-4504-46bb-bcfa-db17e680bae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:26.064553 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fcacc-4504-46bb-bcfa-db17e680bae6 admin admin] PATH_INFO: `/v3/projects/65bfd9750d3f46e7bf7e6c43741d0a6b` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:26.065685 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20fcacc-4504-46bb-bcfa-db17e680bae6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65bfd9750d3f46e7bf7e6c43741d0a6b)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:26.067679 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20fcacc-4504-46bb-bcfa-db17e680bae6 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:26.071763 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c696b5c-aca3-491c-956f-e2950d0d333b admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:26.091451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6979c38-5119-40d7-99a6-5c74a41ee763 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:26.093253 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6979c38-5119-40d7-99a6-5c74a41ee763 service nova] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:26.094310 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6979c38-5119-40d7-99a6-5c74a41ee763 service nova] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:26.094698 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6979c38-5119-40d7-99a6-5c74a41ee763 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:26.097719 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-c6979c38-5119-40d7-99a6-5c74a41ee763 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:26.168686 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-e20fcacc-4504-46bb-bcfa-db17e680bae6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65bfd9750d3f46e7bf7e6c43741d0a6b', 'request_id': 'req-e20fcacc-4504-46bb-bcfa-db17e680bae6'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:26.188832 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-e20fcacc-4504-46bb-bcfa-db17e680bae6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65bfd9750d3f46e7bf7e6c43741d0a6b', 'request_id': 'req-e20fcacc-4504-46bb-bcfa-db17e680bae6'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:26.200569 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-e20fcacc-4504-46bb-bcfa-db17e680bae6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65bfd9750d3f46e7bf7e6c43741d0a6b', 'request_id': 'req-e20fcacc-4504-46bb-bcfa-db17e680bae6'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:26.222340 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-e20fcacc-4504-46bb-bcfa-db17e680bae6 admin admin] The token cache is being invalidate because project 65bfd9750d3f46e7bf7e6c43741d0a6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:43:26.222831 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-e20fcacc-4504-46bb-bcfa-db17e680bae6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:26.223764 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-e20fcacc-4504-46bb-bcfa-db17e680bae6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:26.228018 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 850/1703] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:43:25 2022] DELETE /identity/v3/projects/65bfd9750d3f46e7bf7e6c43741d0a6b => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:26.231179 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd30655-ff3d-4aff-a12a-326235174a0e None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:26.232674 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd30655-ff3d-4aff-a12a-326235174a0e None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:26.233270 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd30655-ff3d-4aff-a12a-326235174a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:26.250327 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6979c38-5119-40d7-99a6-5c74a41ee763 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:26.252877 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6979c38-5119-40d7-99a6-5c74a41ee763 service nova] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:26.271202 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 854/1704] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:25 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:26.272102 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6979c38-5119-40d7-99a6-5c74a41ee763 service nova] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:26.298163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6979c38-5119-40d7-99a6-5c74a41ee763 service nova] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:26.327709 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-cfd30655-ff3d-4aff-a12a-326235174a0e None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:26.329313 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-cfd30655-ff3d-4aff-a12a-326235174a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:26.392140 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf903bbf-226e-4829-950c-9e47a52deaf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:26.392806 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf903bbf-226e-4829-950c-9e47a52deaf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:26.393086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf903bbf-226e-4829-950c-9e47a52deaf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:26.393502 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf903bbf-226e-4829-950c-9e47a52deaf5 admin admin] PATH_INFO: `/v3/users/ad68253397bc47c986e21b4ac899d6ab` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:26.396964 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf903bbf-226e-4829-950c-9e47a52deaf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad68253397bc47c986e21b4ac899d6ab)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:26.399424 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf903bbf-226e-4829-950c-9e47a52deaf5 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:26.547707 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 851/1705] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:26.646990 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-cf903bbf-226e-4829-950c-9e47a52deaf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad68253397bc47c986e21b4ac899d6ab'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:26.648147 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-cf903bbf-226e-4829-950c-9e47a52deaf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad68253397bc47c986e21b4ac899d6ab'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:26.649557 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-cf903bbf-226e-4829-950c-9e47a52deaf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad68253397bc47c986e21b4ac899d6ab'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:26.672413 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-cf903bbf-226e-4829-950c-9e47a52deaf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad68253397bc47c986e21b4ac899d6ab'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:26.700685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-cf903bbf-226e-4829-950c-9e47a52deaf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad68253397bc47c986e21b4ac899d6ab'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:26.795363 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 855/1706] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:43:26 2022] DELETE /identity/v3/users/ad68253397bc47c986e21b4ac899d6ab => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:27.086748 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd30655-ff3d-4aff-a12a-326235174a0e None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:27.090604 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-cfd30655-ff3d-4aff-a12a-326235174a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:27.157989 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:27.302190 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd30655-ff3d-4aff-a12a-326235174a0e None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:27.431218 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b522dc-655b-48cc-abac-0f13b01b22e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:27.432664 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b522dc-655b-48cc-abac-0f13b01b22e9 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:27.432952 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b522dc-655b-48cc-abac-0f13b01b22e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:27.433228 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b522dc-655b-48cc-abac-0f13b01b22e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:27.439984 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-94b522dc-655b-48cc-abac-0f13b01b22e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:27.643286 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b522dc-655b-48cc-abac-0f13b01b22e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:27.677334 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b522dc-655b-48cc-abac-0f13b01b22e9 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:27.677619 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1db9fe-fda6-4230-bc12-1760b300f518 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:27.678644 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1db9fe-fda6-4230-bc12-1760b300f518 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:27.679133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1db9fe-fda6-4230-bc12-1760b300f518 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:27.681686 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 856/1707] 10.222.0.26 () {54 vars in 912 bytes} [Wed Aug 17 23:43:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:43:27.695118 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15776e30-77e3-4196-985a-1fa9d2f265ab None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:27.695633 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15776e30-77e3-4196-985a-1fa9d2f265ab None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:27.696204 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15776e30-77e3-4196-985a-1fa9d2f265ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:27.702527 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 852/1708] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 616 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:27.721044 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b522dc-655b-48cc-abac-0f13b01b22e9 service neutron] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:27.723905 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-94b522dc-655b-48cc-abac-0f13b01b22e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:27.744920 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b522dc-655b-48cc-abac-0f13b01b22e9 service neutron] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:27.752360 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377433b8-9bd4-4910-9935-0ca39cd9b568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:27.753142 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377433b8-9bd4-4910-9935-0ca39cd9b568 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:27.753523 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377433b8-9bd4-4910-9935-0ca39cd9b568 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:27.753893 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377433b8-9bd4-4910-9935-0ca39cd9b568 admin admin] PATH_INFO: `/v3/projects/b956759927da4fe0b70c011b4d0718d5` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:27.755195 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377433b8-9bd4-4910-9935-0ca39cd9b568 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b956759927da4fe0b70c011b4d0718d5)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:27.757281 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377433b8-9bd4-4910-9935-0ca39cd9b568 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:27.785991 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:27.813784 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-15776e30-77e3-4196-985a-1fa9d2f265ab None None] MFA Rules not processed for user `16e2758f1cd144a98b98d9be0fa46ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:27.817709 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-15776e30-77e3-4196-985a-1fa9d2f265ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:27.837989 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-377433b8-9bd4-4910-9935-0ca39cd9b568 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b956759927da4fe0b70c011b4d0718d5', 'request_id': 'req-377433b8-9bd4-4910-9935-0ca39cd9b568'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:27.847679 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-377433b8-9bd4-4910-9935-0ca39cd9b568 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b956759927da4fe0b70c011b4d0718d5', 'request_id': 'req-377433b8-9bd4-4910-9935-0ca39cd9b568'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:27.861945 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-377433b8-9bd4-4910-9935-0ca39cd9b568 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b956759927da4fe0b70c011b4d0718d5', 'request_id': 'req-377433b8-9bd4-4910-9935-0ca39cd9b568'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:27.908549 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-377433b8-9bd4-4910-9935-0ca39cd9b568 admin admin] The token cache is being invalidate because project b956759927da4fe0b70c011b4d0718d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:43:27.909337 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-377433b8-9bd4-4910-9935-0ca39cd9b568 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:27.911684 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-377433b8-9bd4-4910-9935-0ca39cd9b568 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:27.920285 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 853/1709] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:43:27 2022] DELETE /identity/v3/projects/b956759927da4fe0b70c011b4d0718d5 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:27.922684 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3919b5-1071-48d6-af05-e22a91646826 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:27.922851 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3919b5-1071-48d6-af05-e22a91646826 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:27.923197 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3919b5-1071-48d6-af05-e22a91646826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:27.953912 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 857/1710] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Aug 17 23:43:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:43:27.972400 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15776e30-77e3-4196-985a-1fa9d2f265ab None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:27.974218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-15776e30-77e3-4196-985a-1fa9d2f265ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:27.974687 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-8d3919b5-1071-48d6-af05-e22a91646826 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:27.975937 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-8d3919b5-1071-48d6-af05-e22a91646826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:27.987314 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:28.014023 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15776e30-77e3-4196-985a-1fa9d2f265ab None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:28.019565 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab5e9d-3bfb-4be3-ac8c-dc19eac3da48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:28.020223 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab5e9d-3bfb-4be3-ac8c-dc19eac3da48 service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:28.020505 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab5e9d-3bfb-4be3-ac8c-dc19eac3da48 service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:28.020843 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab5e9d-3bfb-4be3-ac8c-dc19eac3da48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:28.041728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ab5e9d-3bfb-4be3-ac8c-dc19eac3da48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:28.043202 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ab5e9d-3bfb-4be3-ac8c-dc19eac3da48 service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:28.068114 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 858/1711] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:28.111496 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 854/1712] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:28.121702 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118edfb-fec0-4c88-a2b4-f09049c80a9f None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:28.122273 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118edfb-fec0-4c88-a2b4-f09049c80a9f None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:28.122706 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118edfb-fec0-4c88-a2b4-f09049c80a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:28.169591 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-b118edfb-fec0-4c88-a2b4-f09049c80a9f None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:28.171348 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-b118edfb-fec0-4c88-a2b4-f09049c80a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:28.189903 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 859/1713] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:28.194453 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3919b5-1071-48d6-af05-e22a91646826 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:28.217309 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3919b5-1071-48d6-af05-e22a91646826 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:28.222896 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11132afc-81e1-4044-b9b8-0055f7a76567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:28.223773 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11132afc-81e1-4044-b9b8-0055f7a76567 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:28.223883 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11132afc-81e1-4044-b9b8-0055f7a76567 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:28.224446 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11132afc-81e1-4044-b9b8-0055f7a76567 admin admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:28.225485 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11132afc-81e1-4044-b9b8-0055f7a76567 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:43:28.225485 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11132afc-81e1-4044-b9b8-0055f7a76567 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:28.227399 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11132afc-81e1-4044-b9b8-0055f7a76567 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:28.246267 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b118edfb-fec0-4c88-a2b4-f09049c80a9f None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:28.252379 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 855/1714] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:43:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:43:28.259126 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b118edfb-fec0-4c88-a2b4-f09049c80a9f None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:28.267319 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8ae05-b794-4766-9aa2-e1f638e790a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:28.268306 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8ae05-b794-4766-9aa2-e1f638e790a1 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:28.268808 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8ae05-b794-4766-9aa2-e1f638e790a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:28.269603 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8ae05-b794-4766-9aa2-e1f638e790a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:28.270638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b8ae05-b794-4766-9aa2-e1f638e790a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:28.273434 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b8ae05-b794-4766-9aa2-e1f638e790a1 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:28.286139 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11132afc-81e1-4044-b9b8-0055f7a76567 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:28.293345 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-11132afc-81e1-4044-b9b8-0055f7a76567 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:28.358908 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11132afc-81e1-4044-b9b8-0055f7a76567 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:28.377306 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba75b8-2e8b-4447-9f4d-3fe9de5dcdac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:28.377888 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba75b8-2e8b-4447-9f4d-3fe9de5dcdac admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:28.378167 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba75b8-2e8b-4447-9f4d-3fe9de5dcdac admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:28.378439 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba75b8-2e8b-4447-9f4d-3fe9de5dcdac admin admin] PATH_INFO: `/v3/projects/aaf39b11fdbc4995aecfa322126450fe` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:28.380548 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ba75b8-2e8b-4447-9f4d-3fe9de5dcdac admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaf39b11fdbc4995aecfa322126450fe)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:28.382565 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ba75b8-2e8b-4447-9f4d-3fe9de5dcdac admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:28.434695 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 860/1715] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:43:28 2022] POST /identity/v3/projects => generated 379 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:43:28.456049 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b850c-437c-45c0-845a-790b2ddf45e9 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:28.456836 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b850c-437c-45c0-845a-790b2ddf45e9 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:28.457123 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b850c-437c-45c0-845a-790b2ddf45e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:28.541448 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-57ba75b8-2e8b-4447-9f4d-3fe9de5dcdac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaf39b11fdbc4995aecfa322126450fe', 'request_id': 'req-57ba75b8-2e8b-4447-9f4d-3fe9de5dcdac'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:28.555310 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-57ba75b8-2e8b-4447-9f4d-3fe9de5dcdac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaf39b11fdbc4995aecfa322126450fe', 'request_id': 'req-57ba75b8-2e8b-4447-9f4d-3fe9de5dcdac'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:28.577431 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-57ba75b8-2e8b-4447-9f4d-3fe9de5dcdac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaf39b11fdbc4995aecfa322126450fe', 'request_id': 'req-57ba75b8-2e8b-4447-9f4d-3fe9de5dcdac'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:28.617683 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-57ba75b8-2e8b-4447-9f4d-3fe9de5dcdac admin admin] The token cache is being invalidate because project aaf39b11fdbc4995aecfa322126450fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:43:28.618488 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-57ba75b8-2e8b-4447-9f4d-3fe9de5dcdac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:28.620096 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-57ba75b8-2e8b-4447-9f4d-3fe9de5dcdac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:28.627725 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 856/1716] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:43:28 2022] DELETE /identity/v3/projects/aaf39b11fdbc4995aecfa322126450fe => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:28.629081 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba75b8-2e8b-4447-9f4d-3fe9de5dcdac admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:28.630926 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-57ba75b8-2e8b-4447-9f4d-3fe9de5dcdac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:28.642515 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-c32b850c-437c-45c0-845a-790b2ddf45e9 None None] MFA Rules not processed for user `2da686247e7f490ab7924d0456da60d5`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:28.643787 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-c32b850c-437c-45c0-845a-790b2ddf45e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:28.775946 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba75b8-2e8b-4447-9f4d-3fe9de5dcdac admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:28.781254 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e1fa01-9f57-4601-bede-63f10e2369fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:28.781805 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e1fa01-9f57-4601-bede-63f10e2369fb admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:28.782544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e1fa01-9f57-4601-bede-63f10e2369fb admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:28.783886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e1fa01-9f57-4601-bede-63f10e2369fb admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:28.784900 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e1fa01-9f57-4601-bede-63f10e2369fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:28.786934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e1fa01-9f57-4601-bede-63f10e2369fb admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:28.818540 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 861/1717] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:28 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:28.819883 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f03a21-39d0-4992-b8a1-0e3931892b81 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:28.820194 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f03a21-39d0-4992-b8a1-0e3931892b81 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:28.820475 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f03a21-39d0-4992-b8a1-0e3931892b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:28.866660 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 857/1718] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:43:28 2022] POST /identity/v3/users => generated 290 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:28.866660 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e1fa01-9f57-4601-bede-63f10e2369fb admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:28.876882 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-79e1fa01-9f57-4601-bede-63f10e2369fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:28.893250 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-92f03a21-39d0-4992-b8a1-0e3931892b81 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:28.896840 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-92f03a21-39d0-4992-b8a1-0e3931892b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:28.898804 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:28.960672 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 862/1719] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:28.963137 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f03a21-39d0-4992-b8a1-0e3931892b81 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:28.984333 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f03a21-39d0-4992-b8a1-0e3931892b81 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:29.014363 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258540e-b067-4ae8-b935-821851113cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:29.015577 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258540e-b067-4ae8-b935-821851113cd3 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:29.015577 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258540e-b067-4ae8-b935-821851113cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:29.015859 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258540e-b067-4ae8-b935-821851113cd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:29.020443 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e258540e-b067-4ae8-b935-821851113cd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:29.026486 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e258540e-b067-4ae8-b935-821851113cd3 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:29.059531 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e1fa01-9f57-4601-bede-63f10e2369fb admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:29.065041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 863/1720] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:28 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:29.066447 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b4303-00bc-4328-acb9-910d24084e3e None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:29.066733 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b4303-00bc-4328-acb9-910d24084e3e None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:29.067004 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b4303-00bc-4328-acb9-910d24084e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:29.068236 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acac546-7486-44e9-a6b0-2b6b16e49f69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:29.071911 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acac546-7486-44e9-a6b0-2b6b16e49f69 service nova] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:29.072292 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acac546-7486-44e9-a6b0-2b6b16e49f69 service nova] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:29.072574 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acac546-7486-44e9-a6b0-2b6b16e49f69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:29.117524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acac546-7486-44e9-a6b0-2b6b16e49f69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:29.122096 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acac546-7486-44e9-a6b0-2b6b16e49f69 service nova] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:29.155214 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-953b4303-00bc-4328-acb9-910d24084e3e None None] MFA Rules not processed for user `7c852a7d720347be87d8205716dfc15f`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:29.156507 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-953b4303-00bc-4328-acb9-910d24084e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:29.174769 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 858/1721] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:28 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:29.176266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e39db-004c-4ed6-a26d-6f24c986ce99 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:29.176565 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e39db-004c-4ed6-a26d-6f24c986ce99 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:29.176848 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e39db-004c-4ed6-a26d-6f24c986ce99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:29.243410 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-819e39db-004c-4ed6-a26d-6f24c986ce99 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:29.245067 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-819e39db-004c-4ed6-a26d-6f24c986ce99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:29.250536 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:29.265668 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 859/1722] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:29.266878 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e39db-004c-4ed6-a26d-6f24c986ce99 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:29.299232 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e39db-004c-4ed6-a26d-6f24c986ce99 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:29.304763 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6305572-de86-40de-8cf0-f921b2996ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:29.305621 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6305572-de86-40de-8cf0-f921b2996ebb admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:29.306083 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6305572-de86-40de-8cf0-f921b2996ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:29.306569 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6305572-de86-40de-8cf0-f921b2996ebb admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:29.307576 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6305572-de86-40de-8cf0-f921b2996ebb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:29.309417 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6305572-de86-40de-8cf0-f921b2996ebb admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:29.321264 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 860/1723] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:29 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:29.323022 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6305572-de86-40de-8cf0-f921b2996ebb admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:29.330057 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 864/1724] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:29.330912 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b4303-00bc-4328-acb9-910d24084e3e None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:29.335324 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6305572-de86-40de-8cf0-f921b2996ebb admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:29.345624 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5811efd-0686-4afd-8a3d-c2ba5484c3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:29.346391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5811efd-0686-4afd-8a3d-c2ba5484c3de admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:29.346855 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5811efd-0686-4afd-8a3d-c2ba5484c3de admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:29.347226 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5811efd-0686-4afd-8a3d-c2ba5484c3de admin admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:29.348474 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5811efd-0686-4afd-8a3d-c2ba5484c3de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:43:29.348474 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5811efd-0686-4afd-8a3d-c2ba5484c3de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:29.350588 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5811efd-0686-4afd-8a3d-c2ba5484c3de admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:29.359456 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b4303-00bc-4328-acb9-910d24084e3e None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:29.365624 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ece2b2-0266-4601-a847-4caa7b167089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:29.366598 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ece2b2-0266-4601-a847-4caa7b167089 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:29.366715 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 861/1725] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:43:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:43:29.368210 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f5446-8752-4517-9840-847fd27b487d None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:29.368600 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f5446-8752-4517-9840-847fd27b487d None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:29.369017 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f5446-8752-4517-9840-847fd27b487d None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:29.370006 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ece2b2-0266-4601-a847-4caa7b167089 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:29.370292 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ece2b2-0266-4601-a847-4caa7b167089 admin admin] PATH_INFO: `/v3/projects/d4edab98379e4e38be7564c38e3a928e/users/1a18536622c041d98dd218e240922ca3/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:29.371544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 862/1726] 10.222.0.26 () {54 vars in 961 bytes} [Wed Aug 17 23:43:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:43:29.373822 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42639f0-71c2-4254-81d7-f967fa550a29 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:29.374211 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42639f0-71c2-4254-81d7-f967fa550a29 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:29.374614 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42639f0-71c2-4254-81d7-f967fa550a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:29.409423 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ece2b2-0266-4601-a847-4caa7b167089 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4edab98379e4e38be7564c38e3a928e, user_id=1a18536622c041d98dd218e240922ca3, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:29.412092 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-e42639f0-71c2-4254-81d7-f967fa550a29 None None] MFA Rules not processed for user `7c852a7d720347be87d8205716dfc15f`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:29.412430 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ece2b2-0266-4601-a847-4caa7b167089 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:29.430832 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-e42639f0-71c2-4254-81d7-f967fa550a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:29.454182 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 865/1727] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:43:29 2022] PUT /identity/v3/projects/d4edab98379e4e38be7564c38e3a928e/users/1a18536622c041d98dd218e240922ca3/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:29.455185 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ece2b2-0266-4601-a847-4caa7b167089 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:29.456835 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 863/1728] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:29.458312 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59e730-5e86-4387-831f-d47d118da0be None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:29.468293 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59e730-5e86-4387-831f-d47d118da0be None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:29.468735 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59e730-5e86-4387-831f-d47d118da0be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:29.519885 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-5f59e730-5e86-4387-831f-d47d118da0be None None] MFA Rules not processed for user `1fc57d911a56423e96c736bc96270ba8`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:29.523098 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-5f59e730-5e86-4387-831f-d47d118da0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:29.635832 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ece2b2-0266-4601-a847-4caa7b167089 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:29.644740 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9861235d-c592-4620-a83b-0a9ac2760033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:29.646023 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9861235d-c592-4620-a83b-0a9ac2760033 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:29.647115 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9861235d-c592-4620-a83b-0a9ac2760033 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:29.647801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9861235d-c592-4620-a83b-0a9ac2760033 admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:29.649377 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9861235d-c592-4620-a83b-0a9ac2760033 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:29.651820 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9861235d-c592-4620-a83b-0a9ac2760033 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:29.707293 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 864/1729] 10.222.0.26 () {58 vars in 1058 bytes} [Wed Aug 17 23:43:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:29.710036 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5102d3-95bf-411c-8737-244cf66e7c0e None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:29.710328 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5102d3-95bf-411c-8737-244cf66e7c0e None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:29.710600 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5102d3-95bf-411c-8737-244cf66e7c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:29.714869 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 866/1730] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:43:29 2022] POST /identity/v3/projects => generated 407 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:29.716225 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9861235d-c592-4620-a83b-0a9ac2760033 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:29.777597 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-aa5102d3-95bf-411c-8737-244cf66e7c0e None None] MFA Rules not processed for user `1a18536622c041d98dd218e240922ca3`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:29.779387 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-aa5102d3-95bf-411c-8737-244cf66e7c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:29.861943 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:29.899377 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9861235d-c592-4620-a83b-0a9ac2760033 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:29.910363 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8103621-0d7c-4b4b-b739-e1efb48876f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:29.911156 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8103621-0d7c-4b4b-b739-e1efb48876f0 service nova] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:29.911570 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8103621-0d7c-4b4b-b739-e1efb48876f0 service nova] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:29.912141 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8103621-0d7c-4b4b-b739-e1efb48876f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:29.929340 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 865/1731] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:29.931210 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d8ffc-d4af-484b-ba93-ce9ba41788fa None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:29.932393 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d8ffc-d4af-484b-ba93-ce9ba41788fa None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:29.933273 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d8ffc-d4af-484b-ba93-ce9ba41788fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:29.960779 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-5f0d8ffc-d4af-484b-ba93-ce9ba41788fa None None] MFA Rules not processed for user `c67dec924b7e4f0aa0874e6e21879889`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:29.963244 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-5f0d8ffc-d4af-484b-ba93-ce9ba41788fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:29.982372 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:30.038808 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8103621-0d7c-4b4b-b739-e1efb48876f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:30.040897 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8103621-0d7c-4b4b-b739-e1efb48876f0 service nova] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:30.048615 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 866/1732] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:29 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:30.053131 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d8ffc-d4af-484b-ba93-ce9ba41788fa None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:30.054616 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-5f0d8ffc-d4af-484b-ba93-ce9ba41788fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:30.075650 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 867/1733] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:30.079011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c096b-5e06-4cca-8165-a2c90b939747 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:30.079111 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c096b-5e06-4cca-8165-a2c90b939747 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:30.079363 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c096b-5e06-4cca-8165-a2c90b939747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:30.112606 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-041c096b-5e06-4cca-8165-a2c90b939747 None None] MFA Rules not processed for user `21e06d9bb5a240459ef3298a990a42f2`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:30.114005 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-041c096b-5e06-4cca-8165-a2c90b939747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:30.227358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d8ffc-d4af-484b-ba93-ce9ba41788fa None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:30.242587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa7def-0b0f-4767-8d4b-a38a144a0f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:30.246976 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa7def-0b0f-4767-8d4b-a38a144a0f21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:30.246976 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa7def-0b0f-4767-8d4b-a38a144a0f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:30.246976 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa7def-0b0f-4767-8d4b-a38a144a0f21 admin admin] PATH_INFO: `/v3/projects/59870531b2514c26b0bb43bf5cfc0316` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:30.253709 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa7def-0b0f-4767-8d4b-a38a144a0f21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59870531b2514c26b0bb43bf5cfc0316)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:30.255694 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa7def-0b0f-4767-8d4b-a38a144a0f21 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:30.258467 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 868/1734] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:30.259446 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041c096b-5e06-4cca-8165-a2c90b939747 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:30.371640 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041c096b-5e06-4cca-8165-a2c90b939747 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:30.376946 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-b1fa7def-0b0f-4767-8d4b-a38a144a0f21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59870531b2514c26b0bb43bf5cfc0316', 'request_id': 'req-b1fa7def-0b0f-4767-8d4b-a38a144a0f21'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:30.387399 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-b1fa7def-0b0f-4767-8d4b-a38a144a0f21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59870531b2514c26b0bb43bf5cfc0316', 'request_id': 'req-b1fa7def-0b0f-4767-8d4b-a38a144a0f21'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:30.405545 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-b1fa7def-0b0f-4767-8d4b-a38a144a0f21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59870531b2514c26b0bb43bf5cfc0316', 'request_id': 'req-b1fa7def-0b0f-4767-8d4b-a38a144a0f21'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:30.426180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-b1fa7def-0b0f-4767-8d4b-a38a144a0f21 admin admin] The token cache is being invalidate because project 59870531b2514c26b0bb43bf5cfc0316 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:43:30.426479 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-b1fa7def-0b0f-4767-8d4b-a38a144a0f21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:30.427166 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-b1fa7def-0b0f-4767-8d4b-a38a144a0f21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:30.429011 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 867/1735] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:43:30 2022] DELETE /identity/v3/projects/59870531b2514c26b0bb43bf5cfc0316 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:30.429701 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa7def-0b0f-4767-8d4b-a38a144a0f21 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:30.436955 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-b1fa7def-0b0f-4767-8d4b-a38a144a0f21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:30.483486 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:30.525250 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc999f-7f12-4058-9a95-1ea05e464880 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc57d911a56423e96c736bc96270ba8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:30.525886 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc999f-7f12-4058-9a95-1ea05e464880 service placement] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:30.526164 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc999f-7f12-4058-9a95-1ea05e464880 service placement] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:30.526437 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc999f-7f12-4058-9a95-1ea05e464880 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:30.528564 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-0dfc999f-7f12-4058-9a95-1ea05e464880 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:30.559124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc999f-7f12-4058-9a95-1ea05e464880 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:30.566404 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc999f-7f12-4058-9a95-1ea05e464880 service placement] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:30.574925 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa7def-0b0f-4767-8d4b-a38a144a0f21 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:30.584323 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5677c872-90a7-4f31-86c6-1d181c48b676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:30.585139 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677c872-90a7-4f31-86c6-1d181c48b676 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:30.585317 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677c872-90a7-4f31-86c6-1d181c48b676 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:30.585576 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677c872-90a7-4f31-86c6-1d181c48b676 admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:30.586215 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5677c872-90a7-4f31-86c6-1d181c48b676 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:30.587565 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 869/1736] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:30.589414 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc999f-7f12-4058-9a95-1ea05e464880 service placement] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:30.589919 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5677c872-90a7-4f31-86c6-1d181c48b676 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:30.592744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-0dfc999f-7f12-4058-9a95-1ea05e464880 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:30.624489 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc999f-7f12-4058-9a95-1ea05e464880 service placement] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:30.638123 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41849c1a-c553-4e0e-b6f0-087d8e2cba53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:30.639301 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41849c1a-c553-4e0e-b6f0-087d8e2cba53 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:30.641824 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41849c1a-c553-4e0e-b6f0-087d8e2cba53 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:30.642283 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41849c1a-c553-4e0e-b6f0-087d8e2cba53 admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:30.643005 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41849c1a-c553-4e0e-b6f0-087d8e2cba53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:30.648581 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41849c1a-c553-4e0e-b6f0-087d8e2cba53 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:30.657907 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 868/1737] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:43:30 2022] POST /identity/v3/users => generated 304 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:30.659223 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037266a5-f2eb-4d73-b45d-2ca8158eec65 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:30.659767 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037266a5-f2eb-4d73-b45d-2ca8158eec65 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:30.659886 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:30.664144 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037266a5-f2eb-4d73-b45d-2ca8158eec65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:30.705220 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 870/1738] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:43:30 2022] POST /identity/v3/projects => generated 379 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:30.706103 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41849c1a-c553-4e0e-b6f0-087d8e2cba53 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:30.708721 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-037266a5-f2eb-4d73-b45d-2ca8158eec65 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:30.710044 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-037266a5-f2eb-4d73-b45d-2ca8158eec65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:30.862164 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 869/1739] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:30.863037 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037266a5-f2eb-4d73-b45d-2ca8158eec65 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:30.875021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41849c1a-c553-4e0e-b6f0-087d8e2cba53 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:30.883174 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723bf2d8-9b36-4e48-88fd-8c1a6388c491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:30.884548 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bf2d8-9b36-4e48-88fd-8c1a6388c491 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:30.885456 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bf2d8-9b36-4e48-88fd-8c1a6388c491 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:30.885737 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bf2d8-9b36-4e48-88fd-8c1a6388c491 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:30.886402 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723bf2d8-9b36-4e48-88fd-8c1a6388c491 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:30.888616 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723bf2d8-9b36-4e48-88fd-8c1a6388c491 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:30.898770 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037266a5-f2eb-4d73-b45d-2ca8158eec65 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:30.906126 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 871/1740] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:30.907458 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26822db-f29f-40aa-9b34-9299a6e2f52e None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:30.907750 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26822db-f29f-40aa-9b34-9299a6e2f52e None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:30.908191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26822db-f29f-40aa-9b34-9299a6e2f52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:30.929534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dca76a-639c-48c7-8416-ae68153e7694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:30.930170 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dca76a-639c-48c7-8416-ae68153e7694 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:30.930447 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dca76a-639c-48c7-8416-ae68153e7694 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:30.936666 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dca76a-639c-48c7-8416-ae68153e7694 admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:30.937060 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dca76a-639c-48c7-8416-ae68153e7694 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:30.941262 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dca76a-639c-48c7-8416-ae68153e7694 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:30.967779 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-c26822db-f29f-40aa-9b34-9299a6e2f52e None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:30.969657 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-c26822db-f29f-40aa-9b34-9299a6e2f52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:31.021282 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 872/1741] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:31.022139 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26822db-f29f-40aa-9b34-9299a6e2f52e None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:31.030246 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 870/1742] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:43:30 2022] POST /identity/v3/users => generated 296 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:31.030695 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dca76a-639c-48c7-8416-ae68153e7694 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:31.052900 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26822db-f29f-40aa-9b34-9299a6e2f52e None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:31.068030 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dca76a-639c-48c7-8416-ae68153e7694 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:31.069945 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d329c-6232-4c42-ae6a-d03812129511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:31.070581 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d329c-6232-4c42-ae6a-d03812129511 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:31.070864 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d329c-6232-4c42-ae6a-d03812129511 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:31.075504 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d329c-6232-4c42-ae6a-d03812129511 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:31.078566 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d329c-6232-4c42-ae6a-d03812129511 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:31.084903 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015d2ea2-4227-46ef-9e4b-daa210ca4853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:31.085405 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d329c-6232-4c42-ae6a-d03812129511 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:31.085601 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d2ea2-4227-46ef-9e4b-daa210ca4853 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:31.085815 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d2ea2-4227-46ef-9e4b-daa210ca4853 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:31.086091 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d2ea2-4227-46ef-9e4b-daa210ca4853 admin admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:31.086754 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015d2ea2-4227-46ef-9e4b-daa210ca4853 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:43:31.087059 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015d2ea2-4227-46ef-9e4b-daa210ca4853 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:31.088849 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015d2ea2-4227-46ef-9e4b-daa210ca4853 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:31.108183 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 873/1743] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:31.110271 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d329c-6232-4c42-ae6a-d03812129511 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:31.125139 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d329c-6232-4c42-ae6a-d03812129511 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:31.131336 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a485382c-8e1a-4c3c-a696-6545eefd6010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:31.131986 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485382c-8e1a-4c3c-a696-6545eefd6010 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:31.132291 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485382c-8e1a-4c3c-a696-6545eefd6010 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:31.132569 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485382c-8e1a-4c3c-a696-6545eefd6010 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:31.133454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a485382c-8e1a-4c3c-a696-6545eefd6010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:31.136045 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a485382c-8e1a-4c3c-a696-6545eefd6010 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:31.147696 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 871/1744] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:43:31.150453 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015d2ea2-4227-46ef-9e4b-daa210ca4853 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:31.157443 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 874/1745] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:31.159095 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a485382c-8e1a-4c3c-a696-6545eefd6010 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:31.174982 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a485382c-8e1a-4c3c-a696-6545eefd6010 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:31.180715 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f743360c-c921-499a-b4ba-fc25df75a874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:31.182193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743360c-c921-499a-b4ba-fc25df75a874 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:31.182476 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743360c-c921-499a-b4ba-fc25df75a874 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:31.182903 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743360c-c921-499a-b4ba-fc25df75a874 admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:31.183686 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f743360c-c921-499a-b4ba-fc25df75a874 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:31.185632 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f743360c-c921-499a-b4ba-fc25df75a874 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:31.218934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015d2ea2-4227-46ef-9e4b-daa210ca4853 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:31.253798 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee49e96-2d65-48a4-89d7-f008f22a2a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:31.254469 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee49e96-2d65-48a4-89d7-f008f22a2a2b admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:31.254735 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee49e96-2d65-48a4-89d7-f008f22a2a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:31.255009 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee49e96-2d65-48a4-89d7-f008f22a2a2b admin admin] PATH_INFO: `/v3/projects/8ae2a33811ba414eab2108dc53d36f04/users/bfaacfa96a3a454d8f37ac6a7a6138f4/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:31.272714 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 875/1746] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:43:31 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:31.272714 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f743360c-c921-499a-b4ba-fc25df75a874 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:31.329043 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee49e96-2d65-48a4-89d7-f008f22a2a2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ae2a33811ba414eab2108dc53d36f04, user_id=bfaacfa96a3a454d8f37ac6a7a6138f4, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:31.335644 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee49e96-2d65-48a4-89d7-f008f22a2a2b admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:31.379228 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 872/1747] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:43:31 2022] PUT /identity/v3/projects/8ae2a33811ba414eab2108dc53d36f04/users/bfaacfa96a3a454d8f37ac6a7a6138f4/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:31.380287 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee49e96-2d65-48a4-89d7-f008f22a2a2b admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:31.500956 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f743360c-c921-499a-b4ba-fc25df75a874 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:31.607102 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:31.628920 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee49e96-2d65-48a4-89d7-f008f22a2a2b admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:31.647906 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839f794c-aadd-4e0b-b299-80544b5f1c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:31.648559 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f794c-aadd-4e0b-b299-80544b5f1c4b admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:31.648962 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f794c-aadd-4e0b-b299-80544b5f1c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:31.649337 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f794c-aadd-4e0b-b299-80544b5f1c4b admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:31.650073 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839f794c-aadd-4e0b-b299-80544b5f1c4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:31.653798 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839f794c-aadd-4e0b-b299-80544b5f1c4b admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:31.658877 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:31.711629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd52793-00fc-4f1e-b0a6-b71837c50891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:31.712599 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd52793-00fc-4f1e-b0a6-b71837c50891 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:31.713457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd52793-00fc-4f1e-b0a6-b71837c50891 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:31.713740 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd52793-00fc-4f1e-b0a6-b71837c50891 admin admin] PATH_INFO: `/v3/projects/315dd000441049079f55eaac6a4add86/users/16ef5db06c5b4d2b85ef9eff8123849b/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:31.714329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 873/1748] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:43:31 2022] POST /identity/v3/users => generated 287 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:31.715890 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9880df-6cde-4f18-9564-ff0e2d7c8778 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:31.721439 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9880df-6cde-4f18-9564-ff0e2d7c8778 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:31.722512 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9880df-6cde-4f18-9564-ff0e2d7c8778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:31.752833 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd52793-00fc-4f1e-b0a6-b71837c50891 admin admin] RBAC: Authorizing `identity:create_grant(project_id=315dd000441049079f55eaac6a4add86, user_id=16ef5db06c5b4d2b85ef9eff8123849b, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:31.779365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd52793-00fc-4f1e-b0a6-b71837c50891 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:31.795261 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-8e9880df-6cde-4f18-9564-ff0e2d7c8778 None None] MFA Rules not processed for user `bfaacfa96a3a454d8f37ac6a7a6138f4`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:31.797938 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-8e9880df-6cde-4f18-9564-ff0e2d7c8778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:31.842843 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 876/1749] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:43:31 2022] PUT /identity/v3/projects/315dd000441049079f55eaac6a4add86/users/16ef5db06c5b4d2b85ef9eff8123849b/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:31.843862 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd52793-00fc-4f1e-b0a6-b71837c50891 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:32.103109 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd52793-00fc-4f1e-b0a6-b71837c50891 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:32.130539 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea878710-5352-4bd8-ae0f-242736cf5d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:32.131188 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea878710-5352-4bd8-ae0f-242736cf5d18 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:32.135697 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea878710-5352-4bd8-ae0f-242736cf5d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:32.136109 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 874/1750] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:31 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 421 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:32.137415 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d7a5b-38a6-4a4c-8049-407837b0da27 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:32.137804 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d7a5b-38a6-4a4c-8049-407837b0da27 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:32.137960 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d7a5b-38a6-4a4c-8049-407837b0da27 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:32.138958 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea878710-5352-4bd8-ae0f-242736cf5d18 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:32.139211 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 875/1751] 10.222.0.26 () {54 vars in 912 bytes} [Wed Aug 17 23:43:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:43:32.142434 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea878710-5352-4bd8-ae0f-242736cf5d18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:32.154656 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d7a5b-38a6-4a4c-8049-407837b0da27 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:32.171360 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea878710-5352-4bd8-ae0f-242736cf5d18 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:32.197339 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d7a5b-38a6-4a4c-8049-407837b0da27 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:32.221846 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac7ff69-97a7-41bb-ab2b-044beb497462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:32.222469 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7ff69-97a7-41bb-ab2b-044beb497462 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:32.222746 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7ff69-97a7-41bb-ab2b-044beb497462 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:32.223019 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7ff69-97a7-41bb-ab2b-044beb497462 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:32.223922 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 877/1752] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:32.224908 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea878710-5352-4bd8-ae0f-242736cf5d18 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:32.228952 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac7ff69-97a7-41bb-ab2b-044beb497462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:32.230647 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac7ff69-97a7-41bb-ab2b-044beb497462 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:32.261217 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea878710-5352-4bd8-ae0f-242736cf5d18 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:32.269941 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 876/1753] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:32.269941 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706465ba-7577-4e45-b4f9-ec7770edee51 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:32.269941 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706465ba-7577-4e45-b4f9-ec7770edee51 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:32.269941 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706465ba-7577-4e45-b4f9-ec7770edee51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:32.285884 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5f519d-0ffc-4954-8556-f1831a159feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:32.298609 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5f519d-0ffc-4954-8556-f1831a159feb admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:32.299079 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5f519d-0ffc-4954-8556-f1831a159feb admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:32.300800 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5f519d-0ffc-4954-8556-f1831a159feb admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:32.301655 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5f519d-0ffc-4954-8556-f1831a159feb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:32.315314 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5f519d-0ffc-4954-8556-f1831a159feb admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:32.341299 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-706465ba-7577-4e45-b4f9-ec7770edee51 None None] MFA Rules not processed for user `16e2758f1cd144a98b98d9be0fa46ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:32.352081 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-706465ba-7577-4e45-b4f9-ec7770edee51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:32.424346 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 878/1754] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:43:32 2022] POST /identity/v3/projects => generated 407 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:32.425695 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5f519d-0ffc-4954-8556-f1831a159feb admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:32.558146 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 877/1755] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Aug 17 23:43:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:32.560886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706465ba-7577-4e45-b4f9-ec7770edee51 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:32.594584 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706465ba-7577-4e45-b4f9-ec7770edee51 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:32.603912 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5f519d-0ffc-4954-8556-f1831a159feb admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:32.609549 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86784a9-8034-4880-abab-0ce9fc4aa6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:32.614776 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86784a9-8034-4880-abab-0ce9fc4aa6fe admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:32.615097 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86784a9-8034-4880-abab-0ce9fc4aa6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:32.616366 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369fdaa-5838-4a3d-a53b-1c67ea931740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:32.616500 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86784a9-8034-4880-abab-0ce9fc4aa6fe admin admin] PATH_INFO: `/v3/projects/315dd000441049079f55eaac6a4add86/users/16ef5db06c5b4d2b85ef9eff8123849b/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:32.618716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369fdaa-5838-4a3d-a53b-1c67ea931740 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:32.618822 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369fdaa-5838-4a3d-a53b-1c67ea931740 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:32.619158 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369fdaa-5838-4a3d-a53b-1c67ea931740 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:32.619842 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a369fdaa-5838-4a3d-a53b-1c67ea931740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:32.621846 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86784a9-8034-4880-abab-0ce9fc4aa6fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=315dd000441049079f55eaac6a4add86, user_id=16ef5db06c5b4d2b85ef9eff8123849b, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:32.625223 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a369fdaa-5838-4a3d-a53b-1c67ea931740 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:32.625565 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86784a9-8034-4880-abab-0ce9fc4aa6fe admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:32.655087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 879/1756] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:32.656182 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369fdaa-5838-4a3d-a53b-1c67ea931740 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:32.667784 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369fdaa-5838-4a3d-a53b-1c67ea931740 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:32.668915 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 878/1757] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:43:32 2022] PUT /identity/v3/projects/315dd000441049079f55eaac6a4add86/users/16ef5db06c5b4d2b85ef9eff8123849b/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:32.672426 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86784a9-8034-4880-abab-0ce9fc4aa6fe admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:32.678576 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-c86784a9-8034-4880-abab-0ce9fc4aa6fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:32.798516 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:32.882791 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86784a9-8034-4880-abab-0ce9fc4aa6fe admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:32.892674 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78bd321-77cb-4885-8bd3-10b728592985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:32.893255 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df6292-dfb0-42ab-8de9-6bc4fa47c5d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:32.893868 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df6292-dfb0-42ab-8de9-6bc4fa47c5d3 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:32.894151 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df6292-dfb0-42ab-8de9-6bc4fa47c5d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:32.894426 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df6292-dfb0-42ab-8de9-6bc4fa47c5d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:32.895845 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78bd321-77cb-4885-8bd3-10b728592985 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:32.896286 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78bd321-77cb-4885-8bd3-10b728592985 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:32.897727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78bd321-77cb-4885-8bd3-10b728592985 admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:32.898533 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78bd321-77cb-4885-8bd3-10b728592985 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:32.902974 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78bd321-77cb-4885-8bd3-10b728592985 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:32.912648 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9df6292-dfb0-42ab-8de9-6bc4fa47c5d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:32.914132 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9df6292-dfb0-42ab-8de9-6bc4fa47c5d3 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:32.921430 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:32.938846 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 879/1758] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:32.939735 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df6292-dfb0-42ab-8de9-6bc4fa47c5d3 service neutron] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:32.973850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 880/1759] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:43:32 2022] POST /identity/v3/users => generated 310 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:32.975704 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a824cfb-a539-412c-8afb-59f75fd5d367 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:32.976678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a824cfb-a539-412c-8afb-59f75fd5d367 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:32.979213 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a824cfb-a539-412c-8afb-59f75fd5d367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:32.983774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df6292-dfb0-42ab-8de9-6bc4fa47c5d3 service neutron] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:32.995662 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572fbff1-2cc4-400e-a3fe-6c3a333010e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:32.996360 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572fbff1-2cc4-400e-a3fe-6c3a333010e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:32.996637 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572fbff1-2cc4-400e-a3fe-6c3a333010e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:32.996911 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572fbff1-2cc4-400e-a3fe-6c3a333010e6 admin admin] PATH_INFO: `/v3/projects/e8f882aac0434f49af89ed8c566c057a/users/f23cd844f473475d8762bd155345f44d/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:33.035783 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572fbff1-2cc4-400e-a3fe-6c3a333010e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8f882aac0434f49af89ed8c566c057a, user_id=f23cd844f473475d8762bd155345f44d, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:33.039368 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572fbff1-2cc4-400e-a3fe-6c3a333010e6 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:33.068927 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-1a824cfb-a539-412c-8afb-59f75fd5d367 None None] MFA Rules not processed for user `16ef5db06c5b4d2b85ef9eff8123849b`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:33.071371 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-1a824cfb-a539-412c-8afb-59f75fd5d367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:33.077461 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 880/1760] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:43:32 2022] PUT /identity/v3/projects/e8f882aac0434f49af89ed8c566c057a/users/f23cd844f473475d8762bd155345f44d/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:33.078411 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572fbff1-2cc4-400e-a3fe-6c3a333010e6 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:33.299065 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572fbff1-2cc4-400e-a3fe-6c3a333010e6 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:33.314754 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990aec70-91af-4d59-a505-ad614c411818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:33.315413 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990aec70-91af-4d59-a505-ad614c411818 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:33.319973 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990aec70-91af-4d59-a505-ad614c411818 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:33.320309 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990aec70-91af-4d59-a505-ad614c411818 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:33.321005 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990aec70-91af-4d59-a505-ad614c411818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:33.323109 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990aec70-91af-4d59-a505-ad614c411818 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:33.365133 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 881/1761] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:33.366867 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d30bc7-e6a9-47d4-a7e6-81ceef98462d None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:33.367163 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d30bc7-e6a9-47d4-a7e6-81ceef98462d None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:33.369953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d30bc7-e6a9-47d4-a7e6-81ceef98462d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:33.370073 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 881/1762] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:32 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:33.371956 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a824cfb-a539-412c-8afb-59f75fd5d367 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:33.402676 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a824cfb-a539-412c-8afb-59f75fd5d367 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:33.414723 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a64f88-51cf-482a-a2ff-1b2fdcf139fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:33.415508 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a64f88-51cf-482a-a2ff-1b2fdcf139fc admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:33.416345 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a64f88-51cf-482a-a2ff-1b2fdcf139fc admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:33.416659 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a64f88-51cf-482a-a2ff-1b2fdcf139fc admin admin] PATH_INFO: `/v3/projects/1cd077cc3e3b4ca99f891a17e7a6f858/users/6fda173a3d344dae8cd9b04476fc436a/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:33.456362 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a64f88-51cf-482a-a2ff-1b2fdcf139fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cd077cc3e3b4ca99f891a17e7a6f858, user_id=6fda173a3d344dae8cd9b04476fc436a, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:33.461062 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a64f88-51cf-482a-a2ff-1b2fdcf139fc admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:33.480930 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-23d30bc7-e6a9-47d4-a7e6-81ceef98462d None None] MFA Rules not processed for user `f23cd844f473475d8762bd155345f44d`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:33.482380 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-23d30bc7-e6a9-47d4-a7e6-81ceef98462d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:33.509835 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 882/1763] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:43:33 2022] PUT /identity/v3/projects/1cd077cc3e3b4ca99f891a17e7a6f858/users/6fda173a3d344dae8cd9b04476fc436a/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:33.511426 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a64f88-51cf-482a-a2ff-1b2fdcf139fc admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:33.516806 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-64a64f88-51cf-482a-a2ff-1b2fdcf139fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:33.665403 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a64f88-51cf-482a-a2ff-1b2fdcf139fc admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:33.671637 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d89d742-7b80-4966-b200-0e9e1c3b8ae4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:33.672645 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89d742-7b80-4966-b200-0e9e1c3b8ae4 service cinder] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:33.673939 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89d742-7b80-4966-b200-0e9e1c3b8ae4 service cinder] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:33.675491 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89d742-7b80-4966-b200-0e9e1c3b8ae4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:33.762709 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:33.778859 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 882/1764] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 413 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:33.779771 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d30bc7-e6a9-47d4-a7e6-81ceef98462d None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:33.813697 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d30bc7-e6a9-47d4-a7e6-81ceef98462d None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:33.825717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d89d742-7b80-4966-b200-0e9e1c3b8ae4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:33.832302 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d89d742-7b80-4966-b200-0e9e1c3b8ae4 service cinder] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:33.863134 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 883/1765] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:33 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:33.867455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d89d742-7b80-4966-b200-0e9e1c3b8ae4 service cinder] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:34.004476 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe334c3-0d69-4320-b699-e58b58f6af2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:34.005117 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe334c3-0d69-4320-b699-e58b58f6af2d admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:34.006523 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe334c3-0d69-4320-b699-e58b58f6af2d admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:34.006828 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe334c3-0d69-4320-b699-e58b58f6af2d admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:34.009345 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe334c3-0d69-4320-b699-e58b58f6af2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:34.013963 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe334c3-0d69-4320-b699-e58b58f6af2d admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:34.020237 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:34.054683 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 883/1766] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:34.055671 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe334c3-0d69-4320-b699-e58b58f6af2d admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:34.089016 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe334c3-0d69-4320-b699-e58b58f6af2d admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:34.101796 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12c5d73-53d6-403b-9c16-e8bbbd41cd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:34.106569 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c5d73-53d6-403b-9c16-e8bbbd41cd4a admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:34.107462 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c5d73-53d6-403b-9c16-e8bbbd41cd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:34.109058 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c5d73-53d6-403b-9c16-e8bbbd41cd4a admin admin] PATH_INFO: `/v3/projects/1cd077cc3e3b4ca99f891a17e7a6f858/users/6fda173a3d344dae8cd9b04476fc436a/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:34.113702 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12c5d73-53d6-403b-9c16-e8bbbd41cd4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cd077cc3e3b4ca99f891a17e7a6f858, user_id=6fda173a3d344dae8cd9b04476fc436a, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:34.119882 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12c5d73-53d6-403b-9c16-e8bbbd41cd4a admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:34.164891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 884/1767] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:43:34 2022] PUT /identity/v3/projects/1cd077cc3e3b4ca99f891a17e7a6f858/users/6fda173a3d344dae8cd9b04476fc436a/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:34.169495 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e399d0-c4f1-41d3-b77b-238fa2bf974c None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:34.169495 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e399d0-c4f1-41d3-b77b-238fa2bf974c None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:34.169495 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e399d0-c4f1-41d3-b77b-238fa2bf974c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:34.173776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d89d742-7b80-4966-b200-0e9e1c3b8ae4 service cinder] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:34.261200 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-c8e399d0-c4f1-41d3-b77b-238fa2bf974c None None] MFA Rules not processed for user `6fda173a3d344dae8cd9b04476fc436a`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:34.262648 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-c8e399d0-c4f1-41d3-b77b-238fa2bf974c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:34.385002 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d3e66-2447-42b7-891d-5b946c07fa6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:34.386868 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d3e66-2447-42b7-891d-5b946c07fa6e admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:34.388701 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d3e66-2447-42b7-891d-5b946c07fa6e admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:34.388994 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d3e66-2447-42b7-891d-5b946c07fa6e admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:34.389957 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1d3e66-2447-42b7-891d-5b946c07fa6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:34.397726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1d3e66-2447-42b7-891d-5b946c07fa6e admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:34.499945 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 885/1768] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:34 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:34.502071 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6aa809-4d72-43cc-a654-51121115a749 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:34.502506 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6aa809-4d72-43cc-a654-51121115a749 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:34.502935 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6aa809-4d72-43cc-a654-51121115a749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:34.534513 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-4e6aa809-4d72-43cc-a654-51121115a749 None None] MFA Rules not processed for user `2da686247e7f490ab7924d0456da60d5`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:34.537218 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-4e6aa809-4d72-43cc-a654-51121115a749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:34.558141 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 884/1769] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:43:33 2022] POST /identity/v3/projects => generated 371 bytes in 693 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:34.565840 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d3e66-2447-42b7-891d-5b946c07fa6e admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:34.706482 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 886/1770] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:34 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:34.710169 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6aa809-4d72-43cc-a654-51121115a749 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:34.718882 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-4e6aa809-4d72-43cc-a654-51121115a749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:34.789350 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d3e66-2447-42b7-891d-5b946c07fa6e admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:34.809261 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35ec87f-4540-4dc4-b722-685a552cb1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:34.810432 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ec87f-4540-4dc4-b722-685a552cb1a8 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:34.810924 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ec87f-4540-4dc4-b722-685a552cb1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:34.811400 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ec87f-4540-4dc4-b722-685a552cb1a8 admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:34.812264 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35ec87f-4540-4dc4-b722-685a552cb1a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:34.814451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35ec87f-4540-4dc4-b722-685a552cb1a8 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:34.849480 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:34.895366 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6aa809-4d72-43cc-a654-51121115a749 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:34.898372 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 885/1771] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:43:34 2022] POST /identity/v3/users => generated 292 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:34.902923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35ec87f-4540-4dc4-b722-685a552cb1a8 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:34.936615 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35ec87f-4540-4dc4-b722-685a552cb1a8 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:34.960743 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256325b8-ebfa-440d-bd2f-a2e52c1a6fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:34.965946 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256325b8-ebfa-440d-bd2f-a2e52c1a6fee admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:34.966658 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256325b8-ebfa-440d-bd2f-a2e52c1a6fee admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:34.967147 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256325b8-ebfa-440d-bd2f-a2e52c1a6fee admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:34.968561 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256325b8-ebfa-440d-bd2f-a2e52c1a6fee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:34.975789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256325b8-ebfa-440d-bd2f-a2e52c1a6fee admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:34.989656 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:35.003718 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 886/1772] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:34 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 23:43:35.007452 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256325b8-ebfa-440d-bd2f-a2e52c1a6fee admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:35.029928 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256325b8-ebfa-440d-bd2f-a2e52c1a6fee admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:35.036508 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45374832-b8bc-42f9-9bfd-24719f89ab5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:35.037561 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45374832-b8bc-42f9-9bfd-24719f89ab5b admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:35.038053 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45374832-b8bc-42f9-9bfd-24719f89ab5b admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:35.038564 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45374832-b8bc-42f9-9bfd-24719f89ab5b admin admin] PATH_INFO: `/v3/projects/1a6272d49e994821b53a12be3fd7497b/users/e9db338e2d544688840ac4a7f3114304/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:35.071626 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45374832-b8bc-42f9-9bfd-24719f89ab5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a6272d49e994821b53a12be3fd7497b, user_id=e9db338e2d544688840ac4a7f3114304, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:35.078017 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45374832-b8bc-42f9-9bfd-24719f89ab5b admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:35.085227 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149da125-38a9-4c24-8ce6-0392d0598bba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:35.088742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149da125-38a9-4c24-8ce6-0392d0598bba service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:35.092437 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149da125-38a9-4c24-8ce6-0392d0598bba service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:35.092857 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149da125-38a9-4c24-8ce6-0392d0598bba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:35.107370 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 887/1773] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:43:35 2022] PUT /identity/v3/projects/1a6272d49e994821b53a12be3fd7497b/users/e9db338e2d544688840ac4a7f3114304/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:43:35.109502 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45374832-b8bc-42f9-9bfd-24719f89ab5b admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:35.248287 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149da125-38a9-4c24-8ce6-0392d0598bba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:35.251031 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149da125-38a9-4c24-8ce6-0392d0598bba service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:35.267584 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45374832-b8bc-42f9-9bfd-24719f89ab5b admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:35.274490 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0581015-1447-4984-b0b2-b2d55704943a tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16ef5db06c5b4d2b85ef9eff8123849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '315dd000441049079f55eaac6a4add86', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:35.276404 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0581015-1447-4984-b0b2-b2d55704943a tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:35.276955 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0581015-1447-4984-b0b2-b2d55704943a tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:35.277546 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0581015-1447-4984-b0b2-b2d55704943a tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:35.279824 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0581015-1447-4984-b0b2-b2d55704943a tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:43:35.281232 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0581015-1447-4984-b0b2-b2d55704943a tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:35.291356 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0581015-1447-4984-b0b2-b2d55704943a tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:35.299552 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 887/1774] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:34 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:35.301550 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149da125-38a9-4c24-8ce6-0392d0598bba service cinder] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:35.342217 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149da125-38a9-4c24-8ce6-0392d0598bba service cinder] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:35.345109 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 888/1775] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:43:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:43:35.348467 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0581015-1447-4984-b0b2-b2d55704943a tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:35.368248 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0581015-1447-4984-b0b2-b2d55704943a tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:35.376082 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f371f94e-95ce-469c-bdc3-4ed01600a423 tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16ef5db06c5b4d2b85ef9eff8123849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '315dd000441049079f55eaac6a4add86', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:35.376772 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f371f94e-95ce-469c-bdc3-4ed01600a423 tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:35.377126 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f371f94e-95ce-469c-bdc3-4ed01600a423 tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:35.377648 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f371f94e-95ce-469c-bdc3-4ed01600a423 tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:35.378359 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f371f94e-95ce-469c-bdc3-4ed01600a423 tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:35.381985 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f371f94e-95ce-469c-bdc3-4ed01600a423 tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:35.450979 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 889/1776] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:43:35 2022] POST /identity/v3/projects => generated 355 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:35.457888 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f371f94e-95ce-469c-bdc3-4ed01600a423 tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:35.527977 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f491281-c6fa-4a41-b1f6-e9d2b62ba212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:35.532478 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f491281-c6fa-4a41-b1f6-e9d2b62ba212 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:35.532831 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f491281-c6fa-4a41-b1f6-e9d2b62ba212 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:35.534484 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f491281-c6fa-4a41-b1f6-e9d2b62ba212 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:35.536716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f491281-c6fa-4a41-b1f6-e9d2b62ba212 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:35.541989 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f491281-c6fa-4a41-b1f6-e9d2b62ba212 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:35.577334 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 888/1777] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:35.579769 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f491281-c6fa-4a41-b1f6-e9d2b62ba212 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:35.595351 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f491281-c6fa-4a41-b1f6-e9d2b62ba212 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:35.628311 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f371f94e-95ce-469c-bdc3-4ed01600a423 tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:35.643876 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdedbd2-7d3d-495f-b58f-73d02bf0df3c tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16ef5db06c5b4d2b85ef9eff8123849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '315dd000441049079f55eaac6a4add86', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:35.644587 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdedbd2-7d3d-495f-b58f-73d02bf0df3c tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:35.644872 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdedbd2-7d3d-495f-b58f-73d02bf0df3c tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:35.645311 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdedbd2-7d3d-495f-b58f-73d02bf0df3c tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:35.648384 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdedbd2-7d3d-495f-b58f-73d02bf0df3c tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:43:35.648865 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdedbd2-7d3d-495f-b58f-73d02bf0df3c tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:35.650564 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdedbd2-7d3d-495f-b58f-73d02bf0df3c tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:35.685132 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 890/1778] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:43:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:43:35.733360 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91462132-4462-40d7-a7da-c9f3af50473a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:35.734758 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91462132-4462-40d7-a7da-c9f3af50473a admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:35.735185 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91462132-4462-40d7-a7da-c9f3af50473a admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:35.735626 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91462132-4462-40d7-a7da-c9f3af50473a admin admin] PATH_INFO: `/v3/projects/1a6272d49e994821b53a12be3fd7497b/users/e9db338e2d544688840ac4a7f3114304/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:35.739292 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91462132-4462-40d7-a7da-c9f3af50473a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a6272d49e994821b53a12be3fd7497b, user_id=e9db338e2d544688840ac4a7f3114304, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:35.743301 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91462132-4462-40d7-a7da-c9f3af50473a admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:35.760715 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdedbd2-7d3d-495f-b58f-73d02bf0df3c tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:35.771237 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdedbd2-7d3d-495f-b58f-73d02bf0df3c tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:35.795494 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 889/1779] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:43:35 2022] PUT /identity/v3/projects/1a6272d49e994821b53a12be3fd7497b/users/e9db338e2d544688840ac4a7f3114304/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:35.805313 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0de0d9-d88d-436f-aefc-5f8feff32af8 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:35.805606 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0de0d9-d88d-436f-aefc-5f8feff32af8 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:35.805881 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0de0d9-d88d-436f-aefc-5f8feff32af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:35.866903 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:35.883653 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:35.915457 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-ea0de0d9-d88d-436f-aefc-5f8feff32af8 None None] MFA Rules not processed for user `e9db338e2d544688840ac4a7f3114304`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:35.917349 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-ea0de0d9-d88d-436f-aefc-5f8feff32af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:35.962319 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81631b0-edd2-4fd5-9f96-326014aeb7a7 tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16ef5db06c5b4d2b85ef9eff8123849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '315dd000441049079f55eaac6a4add86', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:35.965864 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81631b0-edd2-4fd5-9f96-326014aeb7a7 tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:35.966332 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81631b0-edd2-4fd5-9f96-326014aeb7a7 tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:35.966744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81631b0-edd2-4fd5-9f96-326014aeb7a7 tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] PATH_INFO: `/v3/projects/3e0b770f037e461bacf8adb95c3d67f9` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:35.968373 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81631b0-edd2-4fd5-9f96-326014aeb7a7 tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3e0b770f037e461bacf8adb95c3d67f9)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:35.970680 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81631b0-edd2-4fd5-9f96-326014aeb7a7 tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:36.057672 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 890/1780] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:35 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:36.058678 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0de0d9-d88d-436f-aefc-5f8feff32af8 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:36.107246 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-d81631b0-edd2-4fd5-9f96-326014aeb7a7 tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e0b770f037e461bacf8adb95c3d67f9', 'request_id': 'req-d81631b0-edd2-4fd5-9f96-326014aeb7a7'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:36.150874 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-d81631b0-edd2-4fd5-9f96-326014aeb7a7 tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e0b770f037e461bacf8adb95c3d67f9', 'request_id': 'req-d81631b0-edd2-4fd5-9f96-326014aeb7a7'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:36.177557 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-d81631b0-edd2-4fd5-9f96-326014aeb7a7 tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e0b770f037e461bacf8adb95c3d67f9', 'request_id': 'req-d81631b0-edd2-4fd5-9f96-326014aeb7a7'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:36.211231 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0de0d9-d88d-436f-aefc-5f8feff32af8 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:36.216653 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-d81631b0-edd2-4fd5-9f96-326014aeb7a7 tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] The token cache is being invalidate because project 3e0b770f037e461bacf8adb95c3d67f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:43:36.216951 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-d81631b0-edd2-4fd5-9f96-326014aeb7a7 tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:36.219499 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-d81631b0-edd2-4fd5-9f96-326014aeb7a7 tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:36.228031 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 891/1781] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:43:35 2022] DELETE /identity/v3/projects/3e0b770f037e461bacf8adb95c3d67f9 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:36.229002 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81631b0-edd2-4fd5-9f96-326014aeb7a7 tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:36.237172 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-d81631b0-edd2-4fd5-9f96-326014aeb7a7 tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:36.317908 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b83fb34-29be-49a2-90df-58d2928c7db0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:36.319045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83fb34-29be-49a2-90df-58d2928c7db0 service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:36.319420 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83fb34-29be-49a2-90df-58d2928c7db0 service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:36.319713 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83fb34-29be-49a2-90df-58d2928c7db0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:36.322636 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-4b83fb34-29be-49a2-90df-58d2928c7db0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:36.385948 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81631b0-edd2-4fd5-9f96-326014aeb7a7 tempest-VolumeQuotaClassesTest-862541588 tempest-VolumeQuotaClassesTest-862541588-project-admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:36.396504 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95424349-efd2-423b-a343-339bed1da04d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:36.397137 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95424349-efd2-423b-a343-339bed1da04d service cinder] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:36.397417 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95424349-efd2-423b-a343-339bed1da04d service cinder] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:36.397689 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95424349-efd2-423b-a343-339bed1da04d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:36.401334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-95424349-efd2-423b-a343-339bed1da04d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:36.472336 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b83fb34-29be-49a2-90df-58d2928c7db0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:36.473816 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b83fb34-29be-49a2-90df-58d2928c7db0 service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:36.503240 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 891/1782] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:36 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:36.512859 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4cc4d-375d-4cf0-9ad2-68c262e4dc4a None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:36.512975 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4cc4d-375d-4cf0-9ad2-68c262e4dc4a None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:36.513244 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4cc4d-375d-4cf0-9ad2-68c262e4dc4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:36.555296 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95424349-efd2-423b-a343-339bed1da04d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:36.559411 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:36.562208 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95424349-efd2-423b-a343-339bed1da04d service cinder] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:36.599203 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 892/1783] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:36 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:36.677247 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-8aa4cc4d-375d-4cf0-9ad2-68c262e4dc4a None None] MFA Rules not processed for user `d0a13dbffb4a49508b7fb661cf84e803`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:36.684089 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-8aa4cc4d-375d-4cf0-9ad2-68c262e4dc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:36.903771 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 892/1784] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Aug 17 23:43:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:37.204469 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95424349-efd2-423b-a343-339bed1da04d service cinder] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:37.207053 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-95424349-efd2-423b-a343-339bed1da04d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:37.221124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:37.251983 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4cc4d-375d-4cf0-9ad2-68c262e4dc4a None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:37.254428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-8aa4cc4d-375d-4cf0-9ad2-68c262e4dc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:37.293135 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4cc4d-375d-4cf0-9ad2-68c262e4dc4a None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:37.315809 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d468c-450a-41fc-b0f6-d78dd2beb61e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:37.323957 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d468c-450a-41fc-b0f6-d78dd2beb61e service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:37.324353 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d468c-450a-41fc-b0f6-d78dd2beb61e service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:37.325068 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d468c-450a-41fc-b0f6-d78dd2beb61e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:37.331498 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-c18d468c-450a-41fc-b0f6-d78dd2beb61e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:37.588450 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95424349-efd2-423b-a343-339bed1da04d service cinder] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:37.595766 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18d468c-450a-41fc-b0f6-d78dd2beb61e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:37.601437 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18d468c-450a-41fc-b0f6-d78dd2beb61e service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:37.606968 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fcab1f-8545-4efa-8731-75958e1d0a1e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee422ea90557444fabdc9c292865b9b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:37.607893 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fcab1f-8545-4efa-8731-75958e1d0a1e service swift] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:37.608821 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fcab1f-8545-4efa-8731-75958e1d0a1e service swift] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:37.611915 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fcab1f-8545-4efa-8731-75958e1d0a1e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:37.631587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 893/1785] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 23:43:37.636932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d468c-450a-41fc-b0f6-d78dd2beb61e service cinder] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:37.645203 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fcab1f-8545-4efa-8731-75958e1d0a1e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:37.645585 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-c18d468c-450a-41fc-b0f6-d78dd2beb61e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:37.650798 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fcab1f-8545-4efa-8731-75958e1d0a1e service swift] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:37.698729 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 893/1786] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Aug 17 23:43:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 23:43:37.708075 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fcab1f-8545-4efa-8731-75958e1d0a1e service swift] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:37.710406 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-56fcab1f-8545-4efa-8731-75958e1d0a1e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:37.863421 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d468c-450a-41fc-b0f6-d78dd2beb61e service cinder] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:37.869506 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d220ed-3c2e-4603-a085-53a547a71214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:37.870595 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d220ed-3c2e-4603-a085-53a547a71214 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:37.870595 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d220ed-3c2e-4603-a085-53a547a71214 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:37.871197 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d220ed-3c2e-4603-a085-53a547a71214 admin admin] PATH_INFO: `/v3/users/1a18536622c041d98dd218e240922ca3` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:37.873268 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d220ed-3c2e-4603-a085-53a547a71214 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a18536622c041d98dd218e240922ca3)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:37.876542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d220ed-3c2e-4603-a085-53a547a71214 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:37.903787 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:37.957154 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-72d220ed-3c2e-4603-a085-53a547a71214 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a18536622c041d98dd218e240922ca3'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:37.958985 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-72d220ed-3c2e-4603-a085-53a547a71214 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a18536622c041d98dd218e240922ca3'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:37.983142 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-72d220ed-3c2e-4603-a085-53a547a71214 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a18536622c041d98dd218e240922ca3'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:38.013645 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-72d220ed-3c2e-4603-a085-53a547a71214 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a18536622c041d98dd218e240922ca3'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:38.014716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-72d220ed-3c2e-4603-a085-53a547a71214 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a18536622c041d98dd218e240922ca3'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:38.020710 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fcab1f-8545-4efa-8731-75958e1d0a1e service swift] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:38.049746 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71264262-1458-4052-aec7-0d3561f45ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:38.050415 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71264262-1458-4052-aec7-0d3561f45ec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:38.050701 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71264262-1458-4052-aec7-0d3561f45ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:38.050975 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71264262-1458-4052-aec7-0d3561f45ec1 admin admin] PATH_INFO: `/v3/users/e5e31de870fd4def90df97abc9714d38` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:38.052798 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71264262-1458-4052-aec7-0d3561f45ec1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5e31de870fd4def90df97abc9714d38)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:38.054794 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71264262-1458-4052-aec7-0d3561f45ec1 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:38.070128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 894/1787] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:43:37 2022] DELETE /identity/v3/users/1a18536622c041d98dd218e240922ca3 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:38.076849 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99447e50-f0d1-4fb7-a478-68f74f29eb75 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:38.077594 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99447e50-f0d1-4fb7-a478-68f74f29eb75 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:38.078986 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99447e50-f0d1-4fb7-a478-68f74f29eb75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:38.137532 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-99447e50-f0d1-4fb7-a478-68f74f29eb75 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:38.137532 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-99447e50-f0d1-4fb7-a478-68f74f29eb75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:38.192037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-71264262-1458-4052-aec7-0d3561f45ec1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5e31de870fd4def90df97abc9714d38'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:38.192749 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-71264262-1458-4052-aec7-0d3561f45ec1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5e31de870fd4def90df97abc9714d38'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:38.197490 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-71264262-1458-4052-aec7-0d3561f45ec1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5e31de870fd4def90df97abc9714d38'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:38.209234 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-71264262-1458-4052-aec7-0d3561f45ec1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5e31de870fd4def90df97abc9714d38'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:38.225712 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-71264262-1458-4052-aec7-0d3561f45ec1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5e31de870fd4def90df97abc9714d38'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:38.247055 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 895/1788] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:38.254349 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:38.269067 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 894/1789] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:43:37 2022] DELETE /identity/v3/users/e5e31de870fd4def90df97abc9714d38 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:38.271205 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99447e50-f0d1-4fb7-a478-68f74f29eb75 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:38.274199 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf30d5-518e-40e8-b429-36ed79f6071e None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:38.274543 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf30d5-518e-40e8-b429-36ed79f6071e None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:38.274884 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf30d5-518e-40e8-b429-36ed79f6071e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:38.277263 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-99447e50-f0d1-4fb7-a478-68f74f29eb75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:38.318898 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-fdaf30d5-518e-40e8-b429-36ed79f6071e None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:38.320217 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-fdaf30d5-518e-40e8-b429-36ed79f6071e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:38.431092 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99447e50-f0d1-4fb7-a478-68f74f29eb75 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:38.467310 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 895/1790] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:43:38.476927 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf30d5-518e-40e8-b429-36ed79f6071e None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:38.629288 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf30d5-518e-40e8-b429-36ed79f6071e None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:38.654668 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b799993b-de50-48f1-9c34-ff126543d591 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:38.655317 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799993b-de50-48f1-9c34-ff126543d591 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:38.655602 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799993b-de50-48f1-9c34-ff126543d591 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:38.655877 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799993b-de50-48f1-9c34-ff126543d591 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:38.689238 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7306f0-da1e-4a8b-8f3d-2329573e045a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:38.690526 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7306f0-da1e-4a8b-8f3d-2329573e045a service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:38.691133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7306f0-da1e-4a8b-8f3d-2329573e045a service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:38.691817 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7306f0-da1e-4a8b-8f3d-2329573e045a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:38.708269 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:38.714273 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b799993b-de50-48f1-9c34-ff126543d591 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:38.715763 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b799993b-de50-48f1-9c34-ff126543d591 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:38.735179 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7306f0-da1e-4a8b-8f3d-2329573e045a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:38.738165 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7306f0-da1e-4a8b-8f3d-2329573e045a service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:38.765332 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 896/1791] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:38.800440 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 896/1792] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 23:43:41.023216 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b799993b-de50-48f1-9c34-ff126543d591 service neutron] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:41.024761 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-b799993b-de50-48f1-9c34-ff126543d591 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:41.039954 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b799993b-de50-48f1-9c34-ff126543d591 service neutron] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:41.048088 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65725a81-d101-47ce-a7a6-fcb2cd2ac1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:41.048736 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65725a81-d101-47ce-a7a6-fcb2cd2ac1b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:41.049077 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65725a81-d101-47ce-a7a6-fcb2cd2ac1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:41.049411 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65725a81-d101-47ce-a7a6-fcb2cd2ac1b5 admin admin] PATH_INFO: `/v3/projects/d4edab98379e4e38be7564c38e3a928e` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:41.050618 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65725a81-d101-47ce-a7a6-fcb2cd2ac1b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4edab98379e4e38be7564c38e3a928e)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:41.052666 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65725a81-d101-47ce-a7a6-fcb2cd2ac1b5 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:41.065021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7306f0-da1e-4a8b-8f3d-2329573e045a service neutron] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:41.067307 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-6d7306f0-da1e-4a8b-8f3d-2329573e045a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:41.085557 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7306f0-da1e-4a8b-8f3d-2329573e045a service neutron] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:41.091949 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8ec4f-48c3-4094-a102-24d9eae8625f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:41.092589 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8ec4f-48c3-4094-a102-24d9eae8625f admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:41.093079 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8ec4f-48c3-4094-a102-24d9eae8625f admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:41.093498 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8ec4f-48c3-4094-a102-24d9eae8625f admin admin] PATH_INFO: `/v3/projects/ee9232ccb9f74a76a28ca7c758375f1b` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:41.094809 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8ec4f-48c3-4094-a102-24d9eae8625f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee9232ccb9f74a76a28ca7c758375f1b)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:41.096999 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8ec4f-48c3-4094-a102-24d9eae8625f admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:41.107882 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-65725a81-d101-47ce-a7a6-fcb2cd2ac1b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4edab98379e4e38be7564c38e3a928e', 'request_id': 'req-65725a81-d101-47ce-a7a6-fcb2cd2ac1b5'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:41.115196 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-65725a81-d101-47ce-a7a6-fcb2cd2ac1b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4edab98379e4e38be7564c38e3a928e', 'request_id': 'req-65725a81-d101-47ce-a7a6-fcb2cd2ac1b5'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:41.128848 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-65725a81-d101-47ce-a7a6-fcb2cd2ac1b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4edab98379e4e38be7564c38e3a928e', 'request_id': 'req-65725a81-d101-47ce-a7a6-fcb2cd2ac1b5'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:41.145160 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-65725a81-d101-47ce-a7a6-fcb2cd2ac1b5 admin admin] The token cache is being invalidate because project d4edab98379e4e38be7564c38e3a928e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:43:41.145697 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-65725a81-d101-47ce-a7a6-fcb2cd2ac1b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:41.146675 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-65725a81-d101-47ce-a7a6-fcb2cd2ac1b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:41.152663 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 897/1793] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:43:41 2022] DELETE /identity/v3/projects/d4edab98379e4e38be7564c38e3a928e => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:41.153754 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65725a81-d101-47ce-a7a6-fcb2cd2ac1b5 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:41.155230 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-65725a81-d101-47ce-a7a6-fcb2cd2ac1b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:41.206065 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-f4f8ec4f-48c3-4094-a102-24d9eae8625f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee9232ccb9f74a76a28ca7c758375f1b', 'request_id': 'req-f4f8ec4f-48c3-4094-a102-24d9eae8625f'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:41.215938 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-f4f8ec4f-48c3-4094-a102-24d9eae8625f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee9232ccb9f74a76a28ca7c758375f1b', 'request_id': 'req-f4f8ec4f-48c3-4094-a102-24d9eae8625f'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:41.242953 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-f4f8ec4f-48c3-4094-a102-24d9eae8625f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee9232ccb9f74a76a28ca7c758375f1b', 'request_id': 'req-f4f8ec4f-48c3-4094-a102-24d9eae8625f'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:41.265966 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-f4f8ec4f-48c3-4094-a102-24d9eae8625f admin admin] The token cache is being invalidate because project ee9232ccb9f74a76a28ca7c758375f1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:43:41.266165 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-f4f8ec4f-48c3-4094-a102-24d9eae8625f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:41.267599 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-f4f8ec4f-48c3-4094-a102-24d9eae8625f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:41.273181 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 897/1794] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:43:41 2022] DELETE /identity/v3/projects/ee9232ccb9f74a76a28ca7c758375f1b => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:41.275443 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8ec4f-48c3-4094-a102-24d9eae8625f admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:41.276568 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-f4f8ec4f-48c3-4094-a102-24d9eae8625f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:41.289498 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65725a81-d101-47ce-a7a6-fcb2cd2ac1b5 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:41.325451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:41.401568 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8ec4f-48c3-4094-a102-24d9eae8625f admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:41.414612 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e64bb-960b-4e8e-8047-37591de80c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:41.416524 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e64bb-960b-4e8e-8047-37591de80c09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:41.417123 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e64bb-960b-4e8e-8047-37591de80c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:41.417478 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e64bb-960b-4e8e-8047-37591de80c09 admin admin] PATH_INFO: `/v3/users/2a8a021cda844ce1bdbf67510d8345f3` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:41.421892 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0e64bb-960b-4e8e-8047-37591de80c09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a8a021cda844ce1bdbf67510d8345f3)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:41.425694 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0e64bb-960b-4e8e-8047-37591de80c09 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:41.460378 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5637077f-3039-4095-bbed-894017ccf2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:41.461330 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637077f-3039-4095-bbed-894017ccf2b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:41.462200 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637077f-3039-4095-bbed-894017ccf2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:41.463234 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637077f-3039-4095-bbed-894017ccf2b1 admin admin] PATH_INFO: `/v3/users/16ef5db06c5b4d2b85ef9eff8123849b` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:41.464724 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5637077f-3039-4095-bbed-894017ccf2b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16ef5db06c5b4d2b85ef9eff8123849b)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:41.468599 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5637077f-3039-4095-bbed-894017ccf2b1 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:41.519901 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:41.526327 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-9e0e64bb-960b-4e8e-8047-37591de80c09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a8a021cda844ce1bdbf67510d8345f3'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:41.527995 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-9e0e64bb-960b-4e8e-8047-37591de80c09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a8a021cda844ce1bdbf67510d8345f3'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:41.529186 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-9e0e64bb-960b-4e8e-8047-37591de80c09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a8a021cda844ce1bdbf67510d8345f3'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:41.540806 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-9e0e64bb-960b-4e8e-8047-37591de80c09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a8a021cda844ce1bdbf67510d8345f3'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:41.548616 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-5637077f-3039-4095-bbed-894017ccf2b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16ef5db06c5b4d2b85ef9eff8123849b'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:41.549566 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-5637077f-3039-4095-bbed-894017ccf2b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16ef5db06c5b4d2b85ef9eff8123849b'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:41.552165 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-9e0e64bb-960b-4e8e-8047-37591de80c09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a8a021cda844ce1bdbf67510d8345f3'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:41.563523 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-5637077f-3039-4095-bbed-894017ccf2b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16ef5db06c5b4d2b85ef9eff8123849b'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:41.573421 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-5637077f-3039-4095-bbed-894017ccf2b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16ef5db06c5b4d2b85ef9eff8123849b'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:41.574103 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-5637077f-3039-4095-bbed-894017ccf2b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16ef5db06c5b4d2b85ef9eff8123849b'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:41.574951 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 898/1795] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:43:41 2022] DELETE /identity/v3/users/2a8a021cda844ce1bdbf67510d8345f3 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:41.591317 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 898/1796] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:43:41 2022] DELETE /identity/v3/users/16ef5db06c5b4d2b85ef9eff8123849b => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:41.880008 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e64bb-960b-4e8e-8047-37591de80c09 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:41.880163 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5637077f-3039-4095-bbed-894017ccf2b1 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:41.881874 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-5637077f-3039-4095-bbed-894017ccf2b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:41.882346 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-9e0e64bb-960b-4e8e-8047-37591de80c09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:41.981030 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e64bb-960b-4e8e-8047-37591de80c09 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:41.984466 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5637077f-3039-4095-bbed-894017ccf2b1 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:41.986813 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735fae63-c528-418a-baaf-c5c5dbe6e241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:41.987624 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735fae63-c528-418a-baaf-c5c5dbe6e241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:41.989559 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735fae63-c528-418a-baaf-c5c5dbe6e241 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:41.989996 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735fae63-c528-418a-baaf-c5c5dbe6e241 admin admin] PATH_INFO: `/v3/projects/be508c6c7750485ea29d0d6bc41b22c2` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:41.990106 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5afef08-7d8b-4c1a-b8c1-f3238c84bc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:41.991092 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5afef08-7d8b-4c1a-b8c1-f3238c84bc8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:41.991376 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5afef08-7d8b-4c1a-b8c1-f3238c84bc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:41.991655 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5afef08-7d8b-4c1a-b8c1-f3238c84bc8b admin admin] PATH_INFO: `/v3/projects/315dd000441049079f55eaac6a4add86` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:41.991885 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735fae63-c528-418a-baaf-c5c5dbe6e241 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be508c6c7750485ea29d0d6bc41b22c2)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:41.992861 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5afef08-7d8b-4c1a-b8c1-f3238c84bc8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=315dd000441049079f55eaac6a4add86)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:41.994813 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735fae63-c528-418a-baaf-c5c5dbe6e241 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:41.996062 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5afef08-7d8b-4c1a-b8c1-f3238c84bc8b admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:42.052053 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:42.057507 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-735fae63-c528-418a-baaf-c5c5dbe6e241 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be508c6c7750485ea29d0d6bc41b22c2', 'request_id': 'req-735fae63-c528-418a-baaf-c5c5dbe6e241'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:42.063648 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-735fae63-c528-418a-baaf-c5c5dbe6e241 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be508c6c7750485ea29d0d6bc41b22c2', 'request_id': 'req-735fae63-c528-418a-baaf-c5c5dbe6e241'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:42.084204 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-735fae63-c528-418a-baaf-c5c5dbe6e241 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be508c6c7750485ea29d0d6bc41b22c2', 'request_id': 'req-735fae63-c528-418a-baaf-c5c5dbe6e241'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:42.097328 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-735fae63-c528-418a-baaf-c5c5dbe6e241 admin admin] The token cache is being invalidate because project be508c6c7750485ea29d0d6bc41b22c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:43:42.097620 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-735fae63-c528-418a-baaf-c5c5dbe6e241 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:42.098276 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-735fae63-c528-418a-baaf-c5c5dbe6e241 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:42.099835 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 899/1797] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:43:41 2022] DELETE /identity/v3/projects/be508c6c7750485ea29d0d6bc41b22c2 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:42.104735 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a959a-924f-4d1f-a283-1fea28d26e08 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:42.105021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a959a-924f-4d1f-a283-1fea28d26e08 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:42.105474 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a959a-924f-4d1f-a283-1fea28d26e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:42.108830 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-c5afef08-7d8b-4c1a-b8c1-f3238c84bc8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '315dd000441049079f55eaac6a4add86', 'request_id': 'req-c5afef08-7d8b-4c1a-b8c1-f3238c84bc8b'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:42.114968 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-c5afef08-7d8b-4c1a-b8c1-f3238c84bc8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '315dd000441049079f55eaac6a4add86', 'request_id': 'req-c5afef08-7d8b-4c1a-b8c1-f3238c84bc8b'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:42.123786 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-c5afef08-7d8b-4c1a-b8c1-f3238c84bc8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '315dd000441049079f55eaac6a4add86', 'request_id': 'req-c5afef08-7d8b-4c1a-b8c1-f3238c84bc8b'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:42.135085 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-ac4a959a-924f-4d1f-a283-1fea28d26e08 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:42.136339 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-c5afef08-7d8b-4c1a-b8c1-f3238c84bc8b admin admin] The token cache is being invalidate because project 315dd000441049079f55eaac6a4add86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:43:42.136475 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-ac4a959a-924f-4d1f-a283-1fea28d26e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:42.136694 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-c5afef08-7d8b-4c1a-b8c1-f3238c84bc8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:42.137534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-c5afef08-7d8b-4c1a-b8c1-f3238c84bc8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:42.139242 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 899/1798] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:43:41 2022] DELETE /identity/v3/projects/315dd000441049079f55eaac6a4add86 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:42.144398 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64210ff5-da38-46a0-9332-41d904eafc81 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:42.144687 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64210ff5-da38-46a0-9332-41d904eafc81 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:42.145462 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64210ff5-da38-46a0-9332-41d904eafc81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:42.166258 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-64210ff5-da38-46a0-9332-41d904eafc81 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:42.167469 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-64210ff5-da38-46a0-9332-41d904eafc81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:42.200729 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:42.213341 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 900/1799] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:42.222020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c7a50-7f4e-4b9f-8f14-77070e95f1fb None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:42.222312 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c7a50-7f4e-4b9f-8f14-77070e95f1fb None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:42.222587 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c7a50-7f4e-4b9f-8f14-77070e95f1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:42.240013 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 900/1800] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:42.241621 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a80d58-657c-4070-9ab8-abe4b150da39 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:42.241971 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a80d58-657c-4070-9ab8-abe4b150da39 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:42.242316 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a80d58-657c-4070-9ab8-abe4b150da39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:42.243314 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-957c7a50-7f4e-4b9f-8f14-77070e95f1fb None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:42.245235 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-957c7a50-7f4e-4b9f-8f14-77070e95f1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:42.256013 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 901/1801] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:43:42.257293 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c7a50-7f4e-4b9f-8f14-77070e95f1fb None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:42.264606 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-f1a80d58-657c-4070-9ab8-abe4b150da39 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:42.265862 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-f1a80d58-657c-4070-9ab8-abe4b150da39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:42.267003 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c7a50-7f4e-4b9f-8f14-77070e95f1fb None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:42.271765 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e6452-000d-4ac4-bf46-8ac72847fa03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:42.272479 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e6452-000d-4ac4-bf46-8ac72847fa03 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:42.272971 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e6452-000d-4ac4-bf46-8ac72847fa03 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:42.273382 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e6452-000d-4ac4-bf46-8ac72847fa03 admin admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:42.274039 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e6452-000d-4ac4-bf46-8ac72847fa03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:43:42.274342 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e6452-000d-4ac4-bf46-8ac72847fa03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:42.277854 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 901/1802] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:42.278011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e6452-000d-4ac4-bf46-8ac72847fa03 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:42.278906 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a80d58-657c-4070-9ab8-abe4b150da39 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:42.288755 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a80d58-657c-4070-9ab8-abe4b150da39 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:42.289799 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 902/1803] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:43:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:43:42.294725 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e6452-000d-4ac4-bf46-8ac72847fa03 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:42.296554 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0256ee32-52b1-4f96-9118-da22db8d9daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:42.297166 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0256ee32-52b1-4f96-9118-da22db8d9daf admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:42.297445 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0256ee32-52b1-4f96-9118-da22db8d9daf admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:42.297717 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0256ee32-52b1-4f96-9118-da22db8d9daf admin admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:42.298370 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0256ee32-52b1-4f96-9118-da22db8d9daf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:43:42.298908 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0256ee32-52b1-4f96-9118-da22db8d9daf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:42.301715 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0256ee32-52b1-4f96-9118-da22db8d9daf admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:42.304883 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e6452-000d-4ac4-bf46-8ac72847fa03 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:42.320035 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 902/1804] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:43:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:43:42.324048 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795e7bf-522c-4443-80f1-a370d137212e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:42.324675 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795e7bf-522c-4443-80f1-a370d137212e admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:42.324951 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795e7bf-522c-4443-80f1-a370d137212e admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:42.325224 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795e7bf-522c-4443-80f1-a370d137212e admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:42.325849 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8795e7bf-522c-4443-80f1-a370d137212e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:42.328159 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8795e7bf-522c-4443-80f1-a370d137212e admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:42.338884 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0256ee32-52b1-4f96-9118-da22db8d9daf admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:42.366968 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0256ee32-52b1-4f96-9118-da22db8d9daf admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:42.380037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 903/1805] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:43:42 2022] POST /identity/v3/projects => generated 377 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:43:42.380247 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf266a16-5e24-4406-af85-7405c7bab721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:42.380344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf266a16-5e24-4406-af85-7405c7bab721 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:42.380779 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf266a16-5e24-4406-af85-7405c7bab721 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:42.381064 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf266a16-5e24-4406-af85-7405c7bab721 admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:42.384884 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf266a16-5e24-4406-af85-7405c7bab721 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:42.385069 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795e7bf-522c-4443-80f1-a370d137212e admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:42.385159 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf266a16-5e24-4406-af85-7405c7bab721 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:42.421342 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 903/1806] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:43:42 2022] POST /identity/v3/projects => generated 381 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:42.426033 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf266a16-5e24-4406-af85-7405c7bab721 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:42.480398 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795e7bf-522c-4443-80f1-a370d137212e admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:42.525643 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:42.527004 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf266a16-5e24-4406-af85-7405c7bab721 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:42.533549 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e1c2b-1f09-4430-a4fc-655866db5071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:42.535250 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e1c2b-1f09-4430-a4fc-655866db5071 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:42.535534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e1c2b-1f09-4430-a4fc-655866db5071 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:42.535824 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e1c2b-1f09-4430-a4fc-655866db5071 admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:42.536698 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e1c2b-1f09-4430-a4fc-655866db5071 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:42.539872 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e1c2b-1f09-4430-a4fc-655866db5071 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:42.562068 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91309b3-0c47-491c-b5a5-56f26567f352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:42.562722 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91309b3-0c47-491c-b5a5-56f26567f352 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:42.563209 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91309b3-0c47-491c-b5a5-56f26567f352 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:42.563716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91309b3-0c47-491c-b5a5-56f26567f352 admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:42.564516 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91309b3-0c47-491c-b5a5-56f26567f352 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:42.567473 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91309b3-0c47-491c-b5a5-56f26567f352 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:42.584046 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 904/1807] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:43:42 2022] POST /identity/v3/users => generated 291 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:42.586352 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e1c2b-1f09-4430-a4fc-655866db5071 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:42.602583 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e1c2b-1f09-4430-a4fc-655866db5071 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:42.608698 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6860cae0-1469-4450-af7c-54fface77c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:42.609371 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860cae0-1469-4450-af7c-54fface77c13 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:42.609766 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860cae0-1469-4450-af7c-54fface77c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:42.612114 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860cae0-1469-4450-af7c-54fface77c13 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:42.613309 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6860cae0-1469-4450-af7c-54fface77c13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:42.615152 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6860cae0-1469-4450-af7c-54fface77c13 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:42.626555 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 904/1808] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:43:42 2022] POST /identity/v3/users => generated 289 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:42.631541 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91309b3-0c47-491c-b5a5-56f26567f352 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:42.632315 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 905/1809] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:42.633675 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6860cae0-1469-4450-af7c-54fface77c13 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:42.645867 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6860cae0-1469-4450-af7c-54fface77c13 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:42.650800 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91309b3-0c47-491c-b5a5-56f26567f352 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:42.651650 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e76bb1-c244-40d6-85d0-45e9284f5eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:42.652442 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e76bb1-c244-40d6-85d0-45e9284f5eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:42.652720 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e76bb1-c244-40d6-85d0-45e9284f5eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:42.652992 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e76bb1-c244-40d6-85d0-45e9284f5eb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:42.663985 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e76bb1-c244-40d6-85d0-45e9284f5eb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:42.663985 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e76bb1-c244-40d6-85d0-45e9284f5eb2 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:42.669766 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b27a3f-2f6a-4ef9-aba2-b35535eca5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:42.670684 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b27a3f-2f6a-4ef9-aba2-b35535eca5ca admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:42.671122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b27a3f-2f6a-4ef9-aba2-b35535eca5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:42.671544 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b27a3f-2f6a-4ef9-aba2-b35535eca5ca admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:42.672424 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b27a3f-2f6a-4ef9-aba2-b35535eca5ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:42.673186 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 906/1810] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:42.674072 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e76bb1-c244-40d6-85d0-45e9284f5eb2 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:42.676611 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b27a3f-2f6a-4ef9-aba2-b35535eca5ca admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:42.699323 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e76bb1-c244-40d6-85d0-45e9284f5eb2 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:42.701015 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 905/1811] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 23:43:42.702173 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b27a3f-2f6a-4ef9-aba2-b35535eca5ca admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:42.714316 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6039fb-abc1-4325-aa5f-6104438872db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:42.715212 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6039fb-abc1-4325-aa5f-6104438872db admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:42.715608 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6039fb-abc1-4325-aa5f-6104438872db admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:42.716007 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6039fb-abc1-4325-aa5f-6104438872db admin admin] PATH_INFO: `/v3/projects/cf76f8b2738f480a92be57313e01e6f0/users/84139191d1ea4798b4f1e48b9a1dbfbf/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:42.718910 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b27a3f-2f6a-4ef9-aba2-b35535eca5ca admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:42.725963 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfd8bc2-1e23-40b1-887c-c830322b30a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:42.726597 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd8bc2-1e23-40b1-887c-c830322b30a1 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:42.726881 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd8bc2-1e23-40b1-887c-c830322b30a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:42.727540 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd8bc2-1e23-40b1-887c-c830322b30a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:42.727878 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfd8bc2-1e23-40b1-887c-c830322b30a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:42.730503 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfd8bc2-1e23-40b1-887c-c830322b30a1 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:42.752137 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6039fb-abc1-4325-aa5f-6104438872db admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf76f8b2738f480a92be57313e01e6f0, user_id=84139191d1ea4798b4f1e48b9a1dbfbf, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:42.752353 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 906/1812] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:42.754349 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfd8bc2-1e23-40b1-887c-c830322b30a1 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:42.755217 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6039fb-abc1-4325-aa5f-6104438872db admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:42.781889 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfd8bc2-1e23-40b1-887c-c830322b30a1 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:42.804646 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 907/1813] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:43:42 2022] PUT /identity/v3/projects/cf76f8b2738f480a92be57313e01e6f0/users/84139191d1ea4798b4f1e48b9a1dbfbf/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:42.807743 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7664b43d-a888-4170-b6e5-46282916dbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:42.808488 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7664b43d-a888-4170-b6e5-46282916dbf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:42.808768 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7664b43d-a888-4170-b6e5-46282916dbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:42.809214 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7664b43d-a888-4170-b6e5-46282916dbf2 admin admin] PATH_INFO: `/v3/projects/506962f3cc994fedb5f2f3fbfc4efdb2/users/245604b5ad2244d6b0662fbf2cb02f5f/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:42.816135 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed1a92-b51a-412d-a1f2-f0e047a48ff9 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:42.816644 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed1a92-b51a-412d-a1f2-f0e047a48ff9 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:42.817054 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed1a92-b51a-412d-a1f2-f0e047a48ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:42.836168 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7664b43d-a888-4170-b6e5-46282916dbf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=506962f3cc994fedb5f2f3fbfc4efdb2, user_id=245604b5ad2244d6b0662fbf2cb02f5f, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:42.839477 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7664b43d-a888-4170-b6e5-46282916dbf2 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:42.850442 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-c1ed1a92-b51a-412d-a1f2-f0e047a48ff9 None None] MFA Rules not processed for user `84139191d1ea4798b4f1e48b9a1dbfbf`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:42.851803 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-c1ed1a92-b51a-412d-a1f2-f0e047a48ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:42.889574 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 907/1814] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:43:42 2022] PUT /identity/v3/projects/506962f3cc994fedb5f2f3fbfc4efdb2/users/245604b5ad2244d6b0662fbf2cb02f5f/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:43:42.893193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd98f3-cdea-4076-9ea7-450aa7c0908b None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:42.893193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd98f3-cdea-4076-9ea7-450aa7c0908b None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:42.893193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd98f3-cdea-4076-9ea7-450aa7c0908b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:42.893509 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:42.924533 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-3cfd98f3-cdea-4076-9ea7-450aa7c0908b None None] MFA Rules not processed for user `245604b5ad2244d6b0662fbf2cb02f5f`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:42.925774 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-3cfd98f3-cdea-4076-9ea7-450aa7c0908b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:42.947346 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 908/1815] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:42.952992 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed1a92-b51a-412d-a1f2-f0e047a48ff9 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:43.017491 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 908/1816] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:43.021790 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd98f3-cdea-4076-9ea7-450aa7c0908b None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:43.032089 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed1a92-b51a-412d-a1f2-f0e047a48ff9 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:43.083688 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd98f3-cdea-4076-9ea7-450aa7c0908b None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:43.089274 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ba2de-e7fb-4d35-89ec-0d781a57c5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:43.089769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ba2de-e7fb-4d35-89ec-0d781a57c5a9 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:43.089955 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ba2de-e7fb-4d35-89ec-0d781a57c5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:43.090162 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ba2de-e7fb-4d35-89ec-0d781a57c5a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:43.090774 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ba2de-e7fb-4d35-89ec-0d781a57c5a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:43.091043 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbadb20-26b9-4bcd-bf16-98f73fa17b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:43.091654 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbadb20-26b9-4bcd-bf16-98f73fa17b2f admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:43.091936 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbadb20-26b9-4bcd-bf16-98f73fa17b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:43.092235 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbadb20-26b9-4bcd-bf16-98f73fa17b2f admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:43.092725 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ba2de-e7fb-4d35-89ec-0d781a57c5a9 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:43.092855 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbadb20-26b9-4bcd-bf16-98f73fa17b2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:43.094735 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbadb20-26b9-4bcd-bf16-98f73fa17b2f admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:43.109504 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:43.123917 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 909/1817] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:43:43 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:43:43.127457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ba2de-e7fb-4d35-89ec-0d781a57c5a9 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:43.128185 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 909/1818] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:43:42 2022] POST /identity/v3/projects => generated 383 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:43:43.131629 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbadb20-26b9-4bcd-bf16-98f73fa17b2f admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:43.196373 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ba2de-e7fb-4d35-89ec-0d781a57c5a9 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:43.198881 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbadb20-26b9-4bcd-bf16-98f73fa17b2f admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:43.202959 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f532678-d089-4383-bbd7-a44e6205352d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:43.203800 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f532678-d089-4383-bbd7-a44e6205352d admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:43.204205 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d571def-a5ba-430c-b811-c0ed9ea11a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:43.204401 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f532678-d089-4383-bbd7-a44e6205352d admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:43.205101 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d571def-a5ba-430c-b811-c0ed9ea11a67 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:43.205533 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f532678-d089-4383-bbd7-a44e6205352d admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:43.205640 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d571def-a5ba-430c-b811-c0ed9ea11a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:43.205974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d571def-a5ba-430c-b811-c0ed9ea11a67 admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:43.206541 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f532678-d089-4383-bbd7-a44e6205352d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:43.206806 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d571def-a5ba-430c-b811-c0ed9ea11a67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:43.209432 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d571def-a5ba-430c-b811-c0ed9ea11a67 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:43.209690 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f532678-d089-4383-bbd7-a44e6205352d admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:43.218961 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:43.243202 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 910/1819] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:43:43 2022] POST /identity/v3/users => generated 294 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:43:43.245318 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f532678-d089-4383-bbd7-a44e6205352d admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:43.245974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 910/1820] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:43:43 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:43:43.249797 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d571def-a5ba-430c-b811-c0ed9ea11a67 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:43.262254 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f532678-d089-4383-bbd7-a44e6205352d admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:43.269894 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1430926-3fb3-4bd1-913f-37151132341b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:43.270814 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1430926-3fb3-4bd1-913f-37151132341b admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:43.271273 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1430926-3fb3-4bd1-913f-37151132341b admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:43.271728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1430926-3fb3-4bd1-913f-37151132341b admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:43.273656 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1430926-3fb3-4bd1-913f-37151132341b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:43.276305 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1430926-3fb3-4bd1-913f-37151132341b admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:43.277260 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d571def-a5ba-430c-b811-c0ed9ea11a67 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:43.284881 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b8e38-a2b2-4713-ba61-00aa83f1c581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:43.285514 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b8e38-a2b2-4713-ba61-00aa83f1c581 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:43.285792 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b8e38-a2b2-4713-ba61-00aa83f1c581 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:43.286065 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b8e38-a2b2-4713-ba61-00aa83f1c581 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:43.286727 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1b8e38-a2b2-4713-ba61-00aa83f1c581 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:43.288454 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1b8e38-a2b2-4713-ba61-00aa83f1c581 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:43.289078 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 911/1821] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:43 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 23:43:43.293200 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1430926-3fb3-4bd1-913f-37151132341b admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:43.305235 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 911/1822] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:43 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:43.306837 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b8e38-a2b2-4713-ba61-00aa83f1c581 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:43.314006 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1430926-3fb3-4bd1-913f-37151132341b admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:43.319925 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b8e38-a2b2-4713-ba61-00aa83f1c581 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:43.320640 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5cca8f-1745-4f9e-8384-ffa09690ee54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:43.321476 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5cca8f-1745-4f9e-8384-ffa09690ee54 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:43.323116 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5cca8f-1745-4f9e-8384-ffa09690ee54 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:43.323657 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5cca8f-1745-4f9e-8384-ffa09690ee54 admin admin] PATH_INFO: `/v3/projects/3f07a425cb3542ff8d9e212afb778f58/users/f4187ce485d54c159bfdff2fe384a138/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:43.330438 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e68dd45-4717-40da-a2fb-4bc2b18210d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:43.331076 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e68dd45-4717-40da-a2fb-4bc2b18210d1 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:43.331389 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e68dd45-4717-40da-a2fb-4bc2b18210d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:43.331692 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e68dd45-4717-40da-a2fb-4bc2b18210d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:43.332424 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e68dd45-4717-40da-a2fb-4bc2b18210d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:43.337578 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e68dd45-4717-40da-a2fb-4bc2b18210d1 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:43.343454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5cca8f-1745-4f9e-8384-ffa09690ee54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f07a425cb3542ff8d9e212afb778f58, user_id=f4187ce485d54c159bfdff2fe384a138, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:43.346908 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5cca8f-1745-4f9e-8384-ffa09690ee54 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:43.371374 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 912/1823] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:43 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:43.374459 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 912/1824] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:43:43 2022] PUT /identity/v3/projects/3f07a425cb3542ff8d9e212afb778f58/users/f4187ce485d54c159bfdff2fe384a138/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:43.376633 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e68dd45-4717-40da-a2fb-4bc2b18210d1 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:43.376974 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5cca8f-1745-4f9e-8384-ffa09690ee54 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:43.400997 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e68dd45-4717-40da-a2fb-4bc2b18210d1 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:43.505721 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63301f-5914-4ccc-a74e-0b1c7c1dacc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:43.506926 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63301f-5914-4ccc-a74e-0b1c7c1dacc4 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:43.507210 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63301f-5914-4ccc-a74e-0b1c7c1dacc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:43.507483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63301f-5914-4ccc-a74e-0b1c7c1dacc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:43.508477 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf63301f-5914-4ccc-a74e-0b1c7c1dacc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:43.510658 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf63301f-5914-4ccc-a74e-0b1c7c1dacc4 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:43.511603 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5cca8f-1745-4f9e-8384-ffa09690ee54 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:43.517053 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae578007-31b5-472b-b27e-e272d26f63f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:43.517898 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae578007-31b5-472b-b27e-e272d26f63f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:43.518183 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae578007-31b5-472b-b27e-e272d26f63f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:43.518459 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae578007-31b5-472b-b27e-e272d26f63f4 admin admin] PATH_INFO: `/v3/projects/e7446ac4f108444984a9702688b910dc/users/f30081ccfc5c4a5ba6c8e93863aa00bd/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:43.531084 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 913/1825] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:43 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 23:43:43.532185 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63301f-5914-4ccc-a74e-0b1c7c1dacc4 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:43.533611 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae578007-31b5-472b-b27e-e272d26f63f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7446ac4f108444984a9702688b910dc, user_id=f30081ccfc5c4a5ba6c8e93863aa00bd, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:43.536582 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae578007-31b5-472b-b27e-e272d26f63f4 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:43.543460 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63301f-5914-4ccc-a74e-0b1c7c1dacc4 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:43.548942 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae11cc7-d280-4634-a413-1fba76601bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:43.549701 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae11cc7-d280-4634-a413-1fba76601bbd admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:43.549990 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae11cc7-d280-4634-a413-1fba76601bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:43.550261 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae11cc7-d280-4634-a413-1fba76601bbd admin admin] PATH_INFO: `/v3/projects/3f07a425cb3542ff8d9e212afb778f58/users/f4187ce485d54c159bfdff2fe384a138/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:43.551205 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 913/1826] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:43:43 2022] PUT /identity/v3/projects/e7446ac4f108444984a9702688b910dc/users/f30081ccfc5c4a5ba6c8e93863aa00bd/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:43.552233 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae11cc7-d280-4634-a413-1fba76601bbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f07a425cb3542ff8d9e212afb778f58, user_id=f4187ce485d54c159bfdff2fe384a138, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:43.554943 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae11cc7-d280-4634-a413-1fba76601bbd admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:43.563187 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd4adc-c69a-4146-899f-10527e76a94b None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:43.563478 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd4adc-c69a-4146-899f-10527e76a94b None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:43.563727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd4adc-c69a-4146-899f-10527e76a94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:43.567519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 914/1827] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:43:43 2022] PUT /identity/v3/projects/3f07a425cb3542ff8d9e212afb778f58/users/f4187ce485d54c159bfdff2fe384a138/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:43.579119 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03337e04-6bef-436f-952a-1557b0364e9e None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:43.579400 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03337e04-6bef-436f-952a-1557b0364e9e None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:43.579806 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03337e04-6bef-436f-952a-1557b0364e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:43.593780 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-b4bd4adc-c69a-4146-899f-10527e76a94b None None] MFA Rules not processed for user `f30081ccfc5c4a5ba6c8e93863aa00bd`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:43.594930 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-b4bd4adc-c69a-4146-899f-10527e76a94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:43.609628 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-03337e04-6bef-436f-952a-1557b0364e9e None None] MFA Rules not processed for user `f4187ce485d54c159bfdff2fe384a138`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:43.610808 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-03337e04-6bef-436f-952a-1557b0364e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:43.615383 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:43.673533 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 914/1828] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:43:43.678504 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bd4adc-c69a-4146-899f-10527e76a94b None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:43.702352 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 915/1829] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:43 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:43:43.713997 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03337e04-6bef-436f-952a-1557b0364e9e None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:43.715501 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-03337e04-6bef-436f-952a-1557b0364e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:43.749981 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bd4adc-c69a-4146-899f-10527e76a94b None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:43.754449 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9136cafb-ff54-4960-89b2-e9ba6232cb5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:43.754747 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:43.755053 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136cafb-ff54-4960-89b2-e9ba6232cb5b admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:43.755325 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136cafb-ff54-4960-89b2-e9ba6232cb5b admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:43.755598 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136cafb-ff54-4960-89b2-e9ba6232cb5b admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:43.756272 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9136cafb-ff54-4960-89b2-e9ba6232cb5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:43.758184 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9136cafb-ff54-4960-89b2-e9ba6232cb5b admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:43.773626 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03337e04-6bef-436f-952a-1557b0364e9e None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:43.778125 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f45458-7ca8-4177-bfd5-fb4183522acd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:43.778922 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f45458-7ca8-4177-bfd5-fb4183522acd service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:43.779357 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f45458-7ca8-4177-bfd5-fb4183522acd service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:43.779603 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f45458-7ca8-4177-bfd5-fb4183522acd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:43.787972 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 915/1830] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:43:43 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:43:43.792455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9136cafb-ff54-4960-89b2-e9ba6232cb5b admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:43.847052 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f45458-7ca8-4177-bfd5-fb4183522acd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:43.849941 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f45458-7ca8-4177-bfd5-fb4183522acd service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:43.871560 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 916/1831] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 23:43:43.891528 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9136cafb-ff54-4960-89b2-e9ba6232cb5b admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:43.900944 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf3bbc-c8bc-4319-be65-14963b31cb2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:43.902012 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf3bbc-c8bc-4319-be65-14963b31cb2c admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:43.902459 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf3bbc-c8bc-4319-be65-14963b31cb2c admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:43.902830 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf3bbc-c8bc-4319-be65-14963b31cb2c admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:43.903749 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcf3bbc-c8bc-4319-be65-14963b31cb2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:43.905834 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcf3bbc-c8bc-4319-be65-14963b31cb2c admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:43.930543 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 916/1832] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:43:43 2022] POST /identity/v3/users => generated 298 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:43:43.935242 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f45458-7ca8-4177-bfd5-fb4183522acd service cinder] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:43.946006 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f45458-7ca8-4177-bfd5-fb4183522acd service cinder] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:43.950739 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db436c-a581-4986-a79d-27ca98d2a6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:43.951341 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db436c-a581-4986-a79d-27ca98d2a6d1 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:43.951619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db436c-a581-4986-a79d-27ca98d2a6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:43.951886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db436c-a581-4986-a79d-27ca98d2a6d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:43.952563 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21db436c-a581-4986-a79d-27ca98d2a6d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:43.954302 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21db436c-a581-4986-a79d-27ca98d2a6d1 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:43.966815 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 917/1833] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:43:43.971277 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf3bbc-c8bc-4319-be65-14963b31cb2c admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:43.984537 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf3bbc-c8bc-4319-be65-14963b31cb2c admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:43.989246 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffe3a0-c739-426d-8236-3719591d49fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:43.989898 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffe3a0-c739-426d-8236-3719591d49fd admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:43.990176 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffe3a0-c739-426d-8236-3719591d49fd admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:43.990557 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffe3a0-c739-426d-8236-3719591d49fd admin admin] PATH_INFO: `/v3/projects/40ddea386ca64c8c8f15041f994d75bb/users/5267e5b3d0ae4a57985e72eb27b0ad57/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:44.006272 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ffe3a0-c739-426d-8236-3719591d49fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=40ddea386ca64c8c8f15041f994d75bb, user_id=5267e5b3d0ae4a57985e72eb27b0ad57, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:44.009053 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ffe3a0-c739-426d-8236-3719591d49fd admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:44.022714 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 917/1834] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:43:43 2022] PUT /identity/v3/projects/40ddea386ca64c8c8f15041f994d75bb/users/5267e5b3d0ae4a57985e72eb27b0ad57/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:44.028915 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db436c-a581-4986-a79d-27ca98d2a6d1 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:44.041492 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db436c-a581-4986-a79d-27ca98d2a6d1 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:44.098637 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c776aec-1471-42fd-b8ea-ee2105ee55ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:44.099192 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c776aec-1471-42fd-b8ea-ee2105ee55ad admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:44.099541 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c776aec-1471-42fd-b8ea-ee2105ee55ad admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:44.099853 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c776aec-1471-42fd-b8ea-ee2105ee55ad admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:44.100497 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c776aec-1471-42fd-b8ea-ee2105ee55ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:44.102259 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c776aec-1471-42fd-b8ea-ee2105ee55ad admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:44.113178 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 918/1835] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 23:43:44.118120 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffe3a0-c739-426d-8236-3719591d49fd admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:44.130290 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffe3a0-c739-426d-8236-3719591d49fd admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:44.159071 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38adb4c-5f50-44d2-b29b-53892e2a1dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:44.160064 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38adb4c-5f50-44d2-b29b-53892e2a1dfc admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:44.160353 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38adb4c-5f50-44d2-b29b-53892e2a1dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:44.160628 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38adb4c-5f50-44d2-b29b-53892e2a1dfc admin admin] PATH_INFO: `/v3/projects/40ddea386ca64c8c8f15041f994d75bb/users/5267e5b3d0ae4a57985e72eb27b0ad57/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:44.167735 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38adb4c-5f50-44d2-b29b-53892e2a1dfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=40ddea386ca64c8c8f15041f994d75bb, user_id=5267e5b3d0ae4a57985e72eb27b0ad57, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:44.175247 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38adb4c-5f50-44d2-b29b-53892e2a1dfc admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:44.193016 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:44.201078 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 918/1836] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:43:44 2022] PUT /identity/v3/projects/40ddea386ca64c8c8f15041f994d75bb/users/5267e5b3d0ae4a57985e72eb27b0ad57/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:44.213430 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4c521-fd41-4f82-80a2-4d393ab6bc2f None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:44.213928 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4c521-fd41-4f82-80a2-4d393ab6bc2f None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:44.214017 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4c521-fd41-4f82-80a2-4d393ab6bc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:44.271515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-dcc4c521-fd41-4f82-80a2-4d393ab6bc2f None None] MFA Rules not processed for user `5267e5b3d0ae4a57985e72eb27b0ad57`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:44.273754 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-dcc4c521-fd41-4f82-80a2-4d393ab6bc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:44.366850 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:44.371815 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 919/1837] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:44 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:44.385280 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38adb4c-5f50-44d2-b29b-53892e2a1dfc admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:44.473666 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38adb4c-5f50-44d2-b29b-53892e2a1dfc admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:44.478251 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c2b89-1fe9-47a0-af8a-66b5f4c35ddc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:44.478930 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c2b89-1fe9-47a0-af8a-66b5f4c35ddc service cinder] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:44.479225 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c2b89-1fe9-47a0-af8a-66b5f4c35ddc service cinder] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:44.479486 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c2b89-1fe9-47a0-af8a-66b5f4c35ddc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:44.643283 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c2b89-1fe9-47a0-af8a-66b5f4c35ddc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:44.651134 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c2b89-1fe9-47a0-af8a-66b5f4c35ddc service cinder] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:44.661817 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 919/1838] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:44 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:47.431185 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4c521-fd41-4f82-80a2-4d393ab6bc2f None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:47.445855 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4c521-fd41-4f82-80a2-4d393ab6bc2f None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:47.454350 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b70b4e-5bbc-4a21-9bb3-a99478f965e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:47.454350 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b70b4e-5bbc-4a21-9bb3-a99478f965e2 service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:47.454350 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b70b4e-5bbc-4a21-9bb3-a99478f965e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:47.454350 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b70b4e-5bbc-4a21-9bb3-a99478f965e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:47.531953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b70b4e-5bbc-4a21-9bb3-a99478f965e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:47.533970 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b70b4e-5bbc-4a21-9bb3-a99478f965e2 service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:47.544590 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 920/1839] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:50.662901 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c2b89-1fe9-47a0-af8a-66b5f4c35ddc service cinder] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:50.664824 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-5e2c2b89-1fe9-47a0-af8a-66b5f4c35ddc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:50.689636 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c2b89-1fe9-47a0-af8a-66b5f4c35ddc service cinder] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:50.695128 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5dcc2a-8a79-4445-8211-c466a073c6af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:50.695128 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5dcc2a-8a79-4445-8211-c466a073c6af service cinder] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:50.695128 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5dcc2a-8a79-4445-8211-c466a073c6af service cinder] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:50.695419 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5dcc2a-8a79-4445-8211-c466a073c6af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:50.707659 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5dcc2a-8a79-4445-8211-c466a073c6af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:50.709139 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5dcc2a-8a79-4445-8211-c466a073c6af service cinder] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:50.725239 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 920/1840] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:50 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:57.081249 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346a3f9-7013-4ab8-9190-56a40c35ae6c None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:57.081761 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346a3f9-7013-4ab8-9190-56a40c35ae6c None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:57.082213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346a3f9-7013-4ab8-9190-56a40c35ae6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:57.101821 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-d346a3f9-7013-4ab8-9190-56a40c35ae6c None None] MFA Rules not processed for user `16e2758f1cd144a98b98d9be0fa46ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:57.103152 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-d346a3f9-7013-4ab8-9190-56a40c35ae6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:57.175424 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 921/1841] 10.222.0.18 () {58 vars in 1015 bytes} [Wed Aug 17 23:43:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:57.250859 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5dcc2a-8a79-4445-8211-c466a073c6af service cinder] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:57.255392 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-ab5dcc2a-8a79-4445-8211-c466a073c6af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:57.262247 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346a3f9-7013-4ab8-9190-56a40c35ae6c None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:57.265709 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-d346a3f9-7013-4ab8-9190-56a40c35ae6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:57.282260 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346a3f9-7013-4ab8-9190-56a40c35ae6c None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:57.283370 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5dcc2a-8a79-4445-8211-c466a073c6af service cinder] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:57.288099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d51ae7-7f17-4911-97db-787a253c892a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:57.288384 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a929a4-d7d9-4da9-b76f-de12e84fefb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:57.288711 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d51ae7-7f17-4911-97db-787a253c892a service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:57.288992 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d51ae7-7f17-4911-97db-787a253c892a service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:57.289205 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a929a4-d7d9-4da9-b76f-de12e84fefb8 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:57.289317 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d51ae7-7f17-4911-97db-787a253c892a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:57.289707 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a929a4-d7d9-4da9-b76f-de12e84fefb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:57.290121 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a929a4-d7d9-4da9-b76f-de12e84fefb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:57.298702 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a929a4-d7d9-4da9-b76f-de12e84fefb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:57.299796 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d51ae7-7f17-4911-97db-787a253c892a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:57.300205 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a929a4-d7d9-4da9-b76f-de12e84fefb8 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:57.301092 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d51ae7-7f17-4911-97db-787a253c892a service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:57.310268 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 922/1842] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 23:43:57.311200 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a929a4-d7d9-4da9-b76f-de12e84fefb8 service neutron] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:57.318601 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 921/1843] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:57.322478 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d51ae7-7f17-4911-97db-787a253c892a service neutron] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:57.333911 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a929a4-d7d9-4da9-b76f-de12e84fefb8 service neutron] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:57.338319 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d51ae7-7f17-4911-97db-787a253c892a service neutron] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:57.344203 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc020d-d428-4930-82b5-0ab2bc27dc2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:57.344989 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc020d-d428-4930-82b5-0ab2bc27dc2b service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:57.345424 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc020d-d428-4930-82b5-0ab2bc27dc2b service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:57.345847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc020d-d428-4930-82b5-0ab2bc27dc2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:57.346274 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6c6ab2-cb87-4616-9940-0300f2d51914 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:57.347183 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c6ab2-cb87-4616-9940-0300f2d51914 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:57.347556 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c6ab2-cb87-4616-9940-0300f2d51914 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:57.347934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c6ab2-cb87-4616-9940-0300f2d51914 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:57.365873 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6c6ab2-cb87-4616-9940-0300f2d51914 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:57.367588 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6c6ab2-cb87-4616-9940-0300f2d51914 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:57.369444 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dc020d-d428-4930-82b5-0ab2bc27dc2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:57.373935 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dc020d-d428-4930-82b5-0ab2bc27dc2b service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:57.386493 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 922/1844] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:57.389180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 923/1845] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:57.944614 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc020d-d428-4930-82b5-0ab2bc27dc2b service neutron] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:57.955161 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc020d-d428-4930-82b5-0ab2bc27dc2b service neutron] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:57.959372 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c9ddd-bc9e-4ab8-83be-5e589f02cd52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:57.959566 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c9ddd-bc9e-4ab8-83be-5e589f02cd52 service cinder] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:57.960365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c9ddd-bc9e-4ab8-83be-5e589f02cd52 service cinder] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:57.960365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c9ddd-bc9e-4ab8-83be-5e589f02cd52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:58.080958 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134c9ddd-bc9e-4ab8-83be-5e589f02cd52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:58.084000 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134c9ddd-bc9e-4ab8-83be-5e589f02cd52 service cinder] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:58.114000 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 923/1846] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:57 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:58.387768 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6c6ab2-cb87-4616-9940-0300f2d51914 service neutron] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:58.389364 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-df6c6ab2-cb87-4616-9940-0300f2d51914 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:58.520037 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6c6ab2-cb87-4616-9940-0300f2d51914 service neutron] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:58.531648 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae6f497-486c-4060-99c5-5a54b8d6526b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:58.532323 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6f497-486c-4060-99c5-5a54b8d6526b admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:58.532602 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6f497-486c-4060-99c5-5a54b8d6526b admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:58.532873 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6f497-486c-4060-99c5-5a54b8d6526b admin admin] PATH_INFO: `/v3/users/bfaacfa96a3a454d8f37ac6a7a6138f4` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:58.534069 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae6f497-486c-4060-99c5-5a54b8d6526b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfaacfa96a3a454d8f37ac6a7a6138f4)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:58.538451 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae6f497-486c-4060-99c5-5a54b8d6526b admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:58.581452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:58.629012 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-3ae6f497-486c-4060-99c5-5a54b8d6526b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfaacfa96a3a454d8f37ac6a7a6138f4'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:58.629897 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-3ae6f497-486c-4060-99c5-5a54b8d6526b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfaacfa96a3a454d8f37ac6a7a6138f4'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:58.669220 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-3ae6f497-486c-4060-99c5-5a54b8d6526b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfaacfa96a3a454d8f37ac6a7a6138f4'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:58.682046 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-3ae6f497-486c-4060-99c5-5a54b8d6526b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfaacfa96a3a454d8f37ac6a7a6138f4'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:58.683897 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-3ae6f497-486c-4060-99c5-5a54b8d6526b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfaacfa96a3a454d8f37ac6a7a6138f4'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:43:58.715759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 924/1847] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:43:58 2022] DELETE /identity/v3/users/bfaacfa96a3a454d8f37ac6a7a6138f4 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:43:58.720187 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519853db-4253-4645-841f-a0dbce24fba3 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:58.722100 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519853db-4253-4645-841f-a0dbce24fba3 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:58.722649 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519853db-4253-4645-841f-a0dbce24fba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:58.741562 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77218dc7-acc3-42ad-8f1a-ce81622cd25f None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:58.742174 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77218dc7-acc3-42ad-8f1a-ce81622cd25f None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:58.742604 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77218dc7-acc3-42ad-8f1a-ce81622cd25f None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:58.744547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 925/1848] 10.222.0.26 () {54 vars in 921 bytes} [Wed Aug 17 23:43:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 23:43:58.761220 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-519853db-4253-4645-841f-a0dbce24fba3 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:58.764091 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-519853db-4253-4645-841f-a0dbce24fba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:58.768981 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb3bb7-2f89-47c2-a951-79f6bd28efcf None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:58.769392 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb3bb7-2f89-47c2-a951-79f6bd28efcf None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:58.769814 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb3bb7-2f89-47c2-a951-79f6bd28efcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:58.776847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:43:58.841193 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-cffb3bb7-2f89-47c2-a951-79f6bd28efcf None None] MFA Rules not processed for user `ec1dcbe43b0749c3bcc4af9100d771d1`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:58.842832 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-cffb3bb7-2f89-47c2-a951-79f6bd28efcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:58.898396 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 924/1849] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:43:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:58.940409 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519853db-4253-4645-841f-a0dbce24fba3 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:58.943698 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-519853db-4253-4645-841f-a0dbce24fba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:59.015655 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 926/1850] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Aug 17 23:43:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:43:59.082563 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519853db-4253-4645-841f-a0dbce24fba3 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:59.093117 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20893ef-5cd9-4a82-aef8-77496046aa16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:59.093550 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20893ef-5cd9-4a82-aef8-77496046aa16 service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:59.094793 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20893ef-5cd9-4a82-aef8-77496046aa16 service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:59.096246 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20893ef-5cd9-4a82-aef8-77496046aa16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:59.118105 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20893ef-5cd9-4a82-aef8-77496046aa16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:59.122783 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20893ef-5cd9-4a82-aef8-77496046aa16 service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:59.164973 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 925/1851] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:59.359504 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb3bb7-2f89-47c2-a951-79f6bd28efcf None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:59.361347 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-cffb3bb7-2f89-47c2-a951-79f6bd28efcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:59.375651 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb3bb7-2f89-47c2-a951-79f6bd28efcf None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:59.381779 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7070cbe-73f1-49d9-8673-ba8ac48635b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:59.382493 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7070cbe-73f1-49d9-8673-ba8ac48635b9 service nova] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:59.382865 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7070cbe-73f1-49d9-8673-ba8ac48635b9 service nova] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:59.383207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7070cbe-73f1-49d9-8673-ba8ac48635b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:59.396412 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7070cbe-73f1-49d9-8673-ba8ac48635b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:59.399536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7070cbe-73f1-49d9-8673-ba8ac48635b9 service nova] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:59.413564 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 927/1852] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:43:59.694955 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55878a6a-ede0-4eb9-b972-1ce971775665 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:59.695753 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55878a6a-ede0-4eb9-b972-1ce971775665 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:59.696239 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55878a6a-ede0-4eb9-b972-1ce971775665 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:59.698667 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 926/1853] 10.222.0.26 () {54 vars in 921 bytes} [Wed Aug 17 23:43:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:43:59.737340 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b92d26-0d9c-4ed6-9375-a7798936854b None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:59.737887 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b92d26-0d9c-4ed6-9375-a7798936854b None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:59.738309 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b92d26-0d9c-4ed6-9375-a7798936854b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:59.761184 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-09b92d26-0d9c-4ed6-9375-a7798936854b None None] MFA Rules not processed for user `ec1dcbe43b0749c3bcc4af9100d771d1`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:43:59.762756 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-09b92d26-0d9c-4ed6-9375-a7798936854b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:43:59.773686 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 928/1854] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Aug 17 23:43:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:43:59.815234 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55878a6a-ede0-4eb9-b972-1ce971775665 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:43:59.842276 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55878a6a-ede0-4eb9-b972-1ce971775665 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:43:59.849234 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f97f76-e78a-4c58-b471-8b7d15d9be02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:43:59.849862 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f97f76-e78a-4c58-b471-8b7d15d9be02 service nova] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:43:59.850145 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f97f76-e78a-4c58-b471-8b7d15d9be02 service nova] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:43:59.850422 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f97f76-e78a-4c58-b471-8b7d15d9be02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:43:59.860702 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f97f76-e78a-4c58-b471-8b7d15d9be02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:43:59.862390 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f97f76-e78a-4c58-b471-8b7d15d9be02 service nova] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:43:59.874844 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 927/1855] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:43:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:44:00.468457 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b92d26-0d9c-4ed6-9375-a7798936854b None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:00.470098 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-09b92d26-0d9c-4ed6-9375-a7798936854b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:00.491448 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b92d26-0d9c-4ed6-9375-a7798936854b None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:00.501576 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3999f26b-b1d9-4dd5-8c96-77e2636cddfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:00.502035 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999f26b-b1d9-4dd5-8c96-77e2636cddfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:00.502530 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999f26b-b1d9-4dd5-8c96-77e2636cddfa admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:00.503010 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999f26b-b1d9-4dd5-8c96-77e2636cddfa admin admin] PATH_INFO: `/v3/projects/8ae2a33811ba414eab2108dc53d36f04` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:00.519147 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3999f26b-b1d9-4dd5-8c96-77e2636cddfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ae2a33811ba414eab2108dc53d36f04)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:00.523175 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3999f26b-b1d9-4dd5-8c96-77e2636cddfa admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:00.633795 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-3999f26b-b1d9-4dd5-8c96-77e2636cddfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ae2a33811ba414eab2108dc53d36f04', 'request_id': 'req-3999f26b-b1d9-4dd5-8c96-77e2636cddfa'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:00.639938 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-3999f26b-b1d9-4dd5-8c96-77e2636cddfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ae2a33811ba414eab2108dc53d36f04', 'request_id': 'req-3999f26b-b1d9-4dd5-8c96-77e2636cddfa'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:00.650582 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-3999f26b-b1d9-4dd5-8c96-77e2636cddfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ae2a33811ba414eab2108dc53d36f04', 'request_id': 'req-3999f26b-b1d9-4dd5-8c96-77e2636cddfa'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:00.656128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:44:00.672189 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-3999f26b-b1d9-4dd5-8c96-77e2636cddfa admin admin] The token cache is being invalidate because project 8ae2a33811ba414eab2108dc53d36f04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:44:00.672780 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-3999f26b-b1d9-4dd5-8c96-77e2636cddfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:00.673457 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-3999f26b-b1d9-4dd5-8c96-77e2636cddfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:00.678152 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 929/1856] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:44:00 2022] DELETE /identity/v3/projects/8ae2a33811ba414eab2108dc53d36f04 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:44:00.684610 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f97f76-e78a-4c58-b471-8b7d15d9be02 service nova] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:00.699340 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f97f76-e78a-4c58-b471-8b7d15d9be02 service nova] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:00.770006 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759b767-7451-4d75-a868-aa36bbb574c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:00.770749 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759b767-7451-4d75-a868-aa36bbb574c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:00.771188 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759b767-7451-4d75-a868-aa36bbb574c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:00.771738 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759b767-7451-4d75-a868-aa36bbb574c1 admin admin] PATH_INFO: `/v3/users/6fda173a3d344dae8cd9b04476fc436a` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:00.773287 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3759b767-7451-4d75-a868-aa36bbb574c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fda173a3d344dae8cd9b04476fc436a)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:00.775355 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3759b767-7451-4d75-a868-aa36bbb574c1 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:00.833992 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:44:00.846595 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-3759b767-7451-4d75-a868-aa36bbb574c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fda173a3d344dae8cd9b04476fc436a'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:00.848028 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-3759b767-7451-4d75-a868-aa36bbb574c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fda173a3d344dae8cd9b04476fc436a'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:00.848826 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-3759b767-7451-4d75-a868-aa36bbb574c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fda173a3d344dae8cd9b04476fc436a'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:00.860407 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-3759b767-7451-4d75-a868-aa36bbb574c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fda173a3d344dae8cd9b04476fc436a'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:00.871662 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-3759b767-7451-4d75-a868-aa36bbb574c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fda173a3d344dae8cd9b04476fc436a'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:00.892702 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 928/1857] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:44:00 2022] DELETE /identity/v3/users/6fda173a3d344dae8cd9b04476fc436a => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:44:01.379622 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3999f26b-b1d9-4dd5-8c96-77e2636cddfa admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:01.381337 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-3999f26b-b1d9-4dd5-8c96-77e2636cddfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:01.396513 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3999f26b-b1d9-4dd5-8c96-77e2636cddfa admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:01.453199 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390468d-1f2e-4426-a23f-4962354d7608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:01.453830 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390468d-1f2e-4426-a23f-4962354d7608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:01.454110 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390468d-1f2e-4426-a23f-4962354d7608 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:01.454381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390468d-1f2e-4426-a23f-4962354d7608 admin admin] PATH_INFO: `/v3/projects/1cd077cc3e3b4ca99f891a17e7a6f858` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:01.455509 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d390468d-1f2e-4426-a23f-4962354d7608 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cd077cc3e3b4ca99f891a17e7a6f858)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:01.457468 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d390468d-1f2e-4426-a23f-4962354d7608 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:01.502393 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-d390468d-1f2e-4426-a23f-4962354d7608 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cd077cc3e3b4ca99f891a17e7a6f858', 'request_id': 'req-d390468d-1f2e-4426-a23f-4962354d7608'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:01.507815 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-d390468d-1f2e-4426-a23f-4962354d7608 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cd077cc3e3b4ca99f891a17e7a6f858', 'request_id': 'req-d390468d-1f2e-4426-a23f-4962354d7608'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:01.515608 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-d390468d-1f2e-4426-a23f-4962354d7608 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cd077cc3e3b4ca99f891a17e7a6f858', 'request_id': 'req-d390468d-1f2e-4426-a23f-4962354d7608'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:01.526887 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-d390468d-1f2e-4426-a23f-4962354d7608 admin admin] The token cache is being invalidate because project 1cd077cc3e3b4ca99f891a17e7a6f858 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:44:01.527285 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-d390468d-1f2e-4426-a23f-4962354d7608 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:01.527955 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-d390468d-1f2e-4426-a23f-4962354d7608 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:01.529466 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 930/1858] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:44:01 2022] DELETE /identity/v3/projects/1cd077cc3e3b4ca99f891a17e7a6f858 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:44:01.534213 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e47629-03fd-49f0-8450-660b61f2fbba None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:01.534520 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e47629-03fd-49f0-8450-660b61f2fbba None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:01.535060 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e47629-03fd-49f0-8450-660b61f2fbba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:01.555023 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-a8e47629-03fd-49f0-8450-660b61f2fbba None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:44:01.556280 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-a8e47629-03fd-49f0-8450-660b61f2fbba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:01.617868 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 929/1859] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:44:01.626018 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d7c29-1319-4816-81be-c051ef4978cc None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:01.626301 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d7c29-1319-4816-81be-c051ef4978cc None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:01.626572 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d7c29-1319-4816-81be-c051ef4978cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:01.645645 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-0d8d7c29-1319-4816-81be-c051ef4978cc None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:44:01.646842 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-0d8d7c29-1319-4816-81be-c051ef4978cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:01.654653 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 931/1860] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:44:01.658935 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e47629-03fd-49f0-8450-660b61f2fbba None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:01.669641 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e47629-03fd-49f0-8450-660b61f2fbba None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:01.674302 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dfa1b3-50e9-472c-8f22-c4232da257a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:01.674906 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfa1b3-50e9-472c-8f22-c4232da257a7 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:01.675257 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfa1b3-50e9-472c-8f22-c4232da257a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:01.675531 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfa1b3-50e9-472c-8f22-c4232da257a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:01.676221 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dfa1b3-50e9-472c-8f22-c4232da257a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:44:01.676531 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dfa1b3-50e9-472c-8f22-c4232da257a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:01.678154 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dfa1b3-50e9-472c-8f22-c4232da257a7 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:01.693520 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 930/1861] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:44:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:44:01.695184 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d7c29-1319-4816-81be-c051ef4978cc None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:01.705151 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d7c29-1319-4816-81be-c051ef4978cc None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:01.711264 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0192fbd-840d-4a9c-9288-5ef2f7f7853c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:01.712085 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0192fbd-840d-4a9c-9288-5ef2f7f7853c admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:01.712363 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0192fbd-840d-4a9c-9288-5ef2f7f7853c admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:01.712630 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0192fbd-840d-4a9c-9288-5ef2f7f7853c admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:01.713276 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0192fbd-840d-4a9c-9288-5ef2f7f7853c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:01.715396 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0192fbd-840d-4a9c-9288-5ef2f7f7853c admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:01.751636 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 932/1862] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:44:01 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:44:01.752615 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dfa1b3-50e9-472c-8f22-c4232da257a7 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:01.763193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dfa1b3-50e9-472c-8f22-c4232da257a7 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:01.841693 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c0b18-6e97-464c-b539-6b8a8ae2433e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:01.842308 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c0b18-6e97-464c-b539-6b8a8ae2433e admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:01.842585 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c0b18-6e97-464c-b539-6b8a8ae2433e admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:01.842854 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c0b18-6e97-464c-b539-6b8a8ae2433e admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:01.843481 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16c0b18-6e97-464c-b539-6b8a8ae2433e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:01.845508 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16c0b18-6e97-464c-b539-6b8a8ae2433e admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:01.870229 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 931/1863] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:44:01 2022] POST /identity/v3/users => generated 292 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:44:01.871488 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0192fbd-840d-4a9c-9288-5ef2f7f7853c admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:01.881835 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0192fbd-840d-4a9c-9288-5ef2f7f7853c admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:01.887442 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced62a0-c7b1-4b6e-9264-f492549a015e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:01.888071 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced62a0-c7b1-4b6e-9264-f492549a015e admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:01.888365 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced62a0-c7b1-4b6e-9264-f492549a015e admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:01.888626 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced62a0-c7b1-4b6e-9264-f492549a015e admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:01.889280 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced62a0-c7b1-4b6e-9264-f492549a015e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:01.891027 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced62a0-c7b1-4b6e-9264-f492549a015e admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:01.902415 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 933/1864] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:44:01.906603 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c0b18-6e97-464c-b539-6b8a8ae2433e admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:01.917420 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c0b18-6e97-464c-b539-6b8a8ae2433e admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:01.922413 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d30a760-f8ca-468d-ae77-9a8d3fa832cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:01.923023 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30a760-f8ca-468d-ae77-9a8d3fa832cc admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:01.923305 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30a760-f8ca-468d-ae77-9a8d3fa832cc admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:01.923578 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30a760-f8ca-468d-ae77-9a8d3fa832cc admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:01.924261 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d30a760-f8ca-468d-ae77-9a8d3fa832cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:01.925854 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d30a760-f8ca-468d-ae77-9a8d3fa832cc admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:01.940015 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 932/1865] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:44:01.941360 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced62a0-c7b1-4b6e-9264-f492549a015e admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:01.953098 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced62a0-c7b1-4b6e-9264-f492549a015e admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:01.959356 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631e195c-4775-4a27-8893-b4d6035c9898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:01.960051 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e195c-4775-4a27-8893-b4d6035c9898 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:01.960493 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e195c-4775-4a27-8893-b4d6035c9898 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:01.960826 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e195c-4775-4a27-8893-b4d6035c9898 admin admin] PATH_INFO: `/v3/projects/8afcd4d11183401e9e95ed399d2835c3/users/6f2f2ae709794cb8a8ff112bf2150dc4/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:01.978172 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631e195c-4775-4a27-8893-b4d6035c9898 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8afcd4d11183401e9e95ed399d2835c3, user_id=6f2f2ae709794cb8a8ff112bf2150dc4, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:01.980995 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631e195c-4775-4a27-8893-b4d6035c9898 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:01.995123 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 934/1866] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:44:01 2022] PUT /identity/v3/projects/8afcd4d11183401e9e95ed399d2835c3/users/6f2f2ae709794cb8a8ff112bf2150dc4/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:44:02.006331 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ed17d-981c-4377-a0cd-0ed98246c58e None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:02.006617 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ed17d-981c-4377-a0cd-0ed98246c58e None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:02.006895 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ed17d-981c-4377-a0cd-0ed98246c58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:02.035029 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-607ed17d-981c-4377-a0cd-0ed98246c58e None None] MFA Rules not processed for user `6f2f2ae709794cb8a8ff112bf2150dc4`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:44:02.036404 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-607ed17d-981c-4377-a0cd-0ed98246c58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:02.048340 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:44:02.101921 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 933/1867] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:44:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:44:02.106362 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631e195c-4775-4a27-8893-b4d6035c9898 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:02.132385 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:44:02.172032 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631e195c-4775-4a27-8893-b4d6035c9898 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:02.176645 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960bc4f2-d932-4b56-8cda-bea8fa21fda0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:02.177242 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960bc4f2-d932-4b56-8cda-bea8fa21fda0 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:02.177516 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960bc4f2-d932-4b56-8cda-bea8fa21fda0 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:02.177782 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960bc4f2-d932-4b56-8cda-bea8fa21fda0 admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:02.178408 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960bc4f2-d932-4b56-8cda-bea8fa21fda0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:02.180366 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960bc4f2-d932-4b56-8cda-bea8fa21fda0 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:02.211050 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 935/1868] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:44:02 2022] POST /identity/v3/projects => generated 381 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:44:02.212290 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607ed17d-981c-4377-a0cd-0ed98246c58e None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:02.223828 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607ed17d-981c-4377-a0cd-0ed98246c58e None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:02.287546 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35508cfc-df56-462c-ad30-e008c4340dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:02.288330 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35508cfc-df56-462c-ad30-e008c4340dae admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:02.288614 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35508cfc-df56-462c-ad30-e008c4340dae admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:02.288889 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35508cfc-df56-462c-ad30-e008c4340dae admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:02.289523 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35508cfc-df56-462c-ad30-e008c4340dae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:02.292149 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35508cfc-df56-462c-ad30-e008c4340dae admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:02.316732 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 934/1869] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:44:02 2022] POST /identity/v3/users => generated 297 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:44:02.318821 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960bc4f2-d932-4b56-8cda-bea8fa21fda0 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:02.329044 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960bc4f2-d932-4b56-8cda-bea8fa21fda0 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:02.333588 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a5a877-a4ef-4d08-b740-bf6d46439277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:02.334179 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5a877-a4ef-4d08-b740-bf6d46439277 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:02.334460 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5a877-a4ef-4d08-b740-bf6d46439277 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:02.334731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5a877-a4ef-4d08-b740-bf6d46439277 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:02.335375 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a5a877-a4ef-4d08-b740-bf6d46439277 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:02.337044 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a5a877-a4ef-4d08-b740-bf6d46439277 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:02.347872 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 936/1870] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:44:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:44:02.351833 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35508cfc-df56-462c-ad30-e008c4340dae admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:02.362419 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35508cfc-df56-462c-ad30-e008c4340dae admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:02.367049 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf473558-7574-4a58-b1f2-3a925baaae95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:02.367694 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf473558-7574-4a58-b1f2-3a925baaae95 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:02.367986 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf473558-7574-4a58-b1f2-3a925baaae95 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:02.368280 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf473558-7574-4a58-b1f2-3a925baaae95 admin admin] PATH_INFO: `/v3/projects/befa66a7a70f4c93b35a52ba1ab9082c/users/80d329290b7344959d6eff6df9f263ce/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:02.382730 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf473558-7574-4a58-b1f2-3a925baaae95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=befa66a7a70f4c93b35a52ba1ab9082c, user_id=80d329290b7344959d6eff6df9f263ce, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:02.386000 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf473558-7574-4a58-b1f2-3a925baaae95 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:02.399647 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 935/1871] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:44:02 2022] PUT /identity/v3/projects/befa66a7a70f4c93b35a52ba1ab9082c/users/80d329290b7344959d6eff6df9f263ce/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:44:02.404165 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a5a877-a4ef-4d08-b740-bf6d46439277 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:02.414255 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a5a877-a4ef-4d08-b740-bf6d46439277 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:02.469600 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fce3b29-9a74-48a9-93f8-f6757e50d540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:02.470272 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce3b29-9a74-48a9-93f8-f6757e50d540 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:02.470552 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce3b29-9a74-48a9-93f8-f6757e50d540 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:02.470824 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce3b29-9a74-48a9-93f8-f6757e50d540 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:02.471458 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fce3b29-9a74-48a9-93f8-f6757e50d540 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:02.473093 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fce3b29-9a74-48a9-93f8-f6757e50d540 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:02.482563 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 937/1872] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:44:02 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:44:02.486853 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf473558-7574-4a58-b1f2-3a925baaae95 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:02.497126 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf473558-7574-4a58-b1f2-3a925baaae95 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:02.501603 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9993de-1c1e-4c0f-92a4-e0b6c259e99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:02.502237 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9993de-1c1e-4c0f-92a4-e0b6c259e99c admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:02.502511 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9993de-1c1e-4c0f-92a4-e0b6c259e99c admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:02.502782 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9993de-1c1e-4c0f-92a4-e0b6c259e99c admin admin] PATH_INFO: `/v3/projects/befa66a7a70f4c93b35a52ba1ab9082c/users/80d329290b7344959d6eff6df9f263ce/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:02.504673 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9993de-1c1e-4c0f-92a4-e0b6c259e99c admin admin] RBAC: Authorizing `identity:create_grant(project_id=befa66a7a70f4c93b35a52ba1ab9082c, user_id=80d329290b7344959d6eff6df9f263ce, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:02.507355 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9993de-1c1e-4c0f-92a4-e0b6c259e99c admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:02.517868 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 936/1873] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:44:02 2022] PUT /identity/v3/projects/befa66a7a70f4c93b35a52ba1ab9082c/users/80d329290b7344959d6eff6df9f263ce/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:44:02.528448 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c93098-8667-4d55-970a-b103b5766b6a None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:02.528901 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c93098-8667-4d55-970a-b103b5766b6a None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:02.528987 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c93098-8667-4d55-970a-b103b5766b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:02.556599 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-93c93098-8667-4d55-970a-b103b5766b6a None None] MFA Rules not processed for user `80d329290b7344959d6eff6df9f263ce`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:44:02.557797 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-93c93098-8667-4d55-970a-b103b5766b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:02.627142 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 938/1874] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:44:02 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:44:02.639342 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9993de-1c1e-4c0f-92a4-e0b6c259e99c admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:02.641050 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-9e9993de-1c1e-4c0f-92a4-e0b6c259e99c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:02.696324 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9993de-1c1e-4c0f-92a4-e0b6c259e99c admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:02.700376 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4397a-20de-4c3d-8d3c-21fe997c7cc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:02.700970 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4397a-20de-4c3d-8d3c-21fe997c7cc9 service cinder] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:02.701245 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4397a-20de-4c3d-8d3c-21fe997c7cc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:02.701520 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4397a-20de-4c3d-8d3c-21fe997c7cc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:02.710456 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a4397a-20de-4c3d-8d3c-21fe997c7cc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:02.711879 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a4397a-20de-4c3d-8d3c-21fe997c7cc9 service cinder] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:02.721727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 937/1875] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:44:02 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:44:02.811851 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c93098-8667-4d55-970a-b103b5766b6a None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:02.821500 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c93098-8667-4d55-970a-b103b5766b6a None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:02.825829 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da25bd-2348-49ce-9ac1-7c77fbc91445 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:02.826469 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da25bd-2348-49ce-9ac1-7c77fbc91445 service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:02.826750 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da25bd-2348-49ce-9ac1-7c77fbc91445 service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:02.827022 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da25bd-2348-49ce-9ac1-7c77fbc91445 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:02.840316 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:44:02.903936 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39da25bd-2348-49ce-9ac1-7c77fbc91445 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:02.903936 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39da25bd-2348-49ce-9ac1-7c77fbc91445 service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:02.911604 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 939/1876] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:44:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:44:03.189253 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4397a-20de-4c3d-8d3c-21fe997c7cc9 service cinder] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:03.199455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:44:03.251827 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4397a-20de-4c3d-8d3c-21fe997c7cc9 service cinder] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:03.256404 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c161a66-c32d-42fe-8c41-a46e84af8d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:03.257028 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c161a66-c32d-42fe-8c41-a46e84af8d65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:03.257306 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c161a66-c32d-42fe-8c41-a46e84af8d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:03.257573 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c161a66-c32d-42fe-8c41-a46e84af8d65 admin admin] PATH_INFO: `/v3/users/6f2f2ae709794cb8a8ff112bf2150dc4` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:03.258788 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c161a66-c32d-42fe-8c41-a46e84af8d65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f2f2ae709794cb8a8ff112bf2150dc4)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:03.261275 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c161a66-c32d-42fe-8c41-a46e84af8d65 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:03.319121 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-3c161a66-c32d-42fe-8c41-a46e84af8d65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f2f2ae709794cb8a8ff112bf2150dc4'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:03.319771 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-3c161a66-c32d-42fe-8c41-a46e84af8d65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f2f2ae709794cb8a8ff112bf2150dc4'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:03.320471 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-3c161a66-c32d-42fe-8c41-a46e84af8d65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f2f2ae709794cb8a8ff112bf2150dc4'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:03.327946 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-3c161a66-c32d-42fe-8c41-a46e84af8d65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f2f2ae709794cb8a8ff112bf2150dc4'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:03.335359 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-3c161a66-c32d-42fe-8c41-a46e84af8d65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f2f2ae709794cb8a8ff112bf2150dc4'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:03.356578 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 938/1877] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:44:03 2022] DELETE /identity/v3/users/6f2f2ae709794cb8a8ff112bf2150dc4 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:44:03.358353 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7ff05-0708-4686-a6af-b563db1b5332 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:03.358715 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7ff05-0708-4686-a6af-b563db1b5332 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:03.359061 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7ff05-0708-4686-a6af-b563db1b5332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:03.384142 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-f5d7ff05-0708-4686-a6af-b563db1b5332 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:44:03.385847 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-f5d7ff05-0708-4686-a6af-b563db1b5332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:03.445804 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 940/1878] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:44:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:44:03.456182 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c161a66-c32d-42fe-8c41-a46e84af8d65 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:03.457586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-3c161a66-c32d-42fe-8c41-a46e84af8d65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:03.513688 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c161a66-c32d-42fe-8c41-a46e84af8d65 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:03.517317 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0c2cf-24d3-442b-bfac-71709a9f8757 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:03.517923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0c2cf-24d3-442b-bfac-71709a9f8757 service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:03.518201 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0c2cf-24d3-442b-bfac-71709a9f8757 service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:03.518466 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0c2cf-24d3-442b-bfac-71709a9f8757 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:03.527229 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace0c2cf-24d3-442b-bfac-71709a9f8757 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:03.528677 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace0c2cf-24d3-442b-bfac-71709a9f8757 service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:03.538179 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 939/1879] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:44:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:44:03.800411 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7ff05-0708-4686-a6af-b563db1b5332 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:03.801938 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-f5d7ff05-0708-4686-a6af-b563db1b5332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:03.814920 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7ff05-0708-4686-a6af-b563db1b5332 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:03.820176 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d64ce8-920d-4d86-a0b1-386040af3e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:03.820879 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d64ce8-920d-4d86-a0b1-386040af3e13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:03.821359 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d64ce8-920d-4d86-a0b1-386040af3e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:03.821668 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d64ce8-920d-4d86-a0b1-386040af3e13 admin admin] PATH_INFO: `/v3/projects/8afcd4d11183401e9e95ed399d2835c3` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:03.823610 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d64ce8-920d-4d86-a0b1-386040af3e13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8afcd4d11183401e9e95ed399d2835c3)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:03.825614 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d64ce8-920d-4d86-a0b1-386040af3e13 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:03.870968 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-d2d64ce8-920d-4d86-a0b1-386040af3e13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8afcd4d11183401e9e95ed399d2835c3', 'request_id': 'req-d2d64ce8-920d-4d86-a0b1-386040af3e13'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:03.876279 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-d2d64ce8-920d-4d86-a0b1-386040af3e13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8afcd4d11183401e9e95ed399d2835c3', 'request_id': 'req-d2d64ce8-920d-4d86-a0b1-386040af3e13'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:03.883426 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-d2d64ce8-920d-4d86-a0b1-386040af3e13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8afcd4d11183401e9e95ed399d2835c3', 'request_id': 'req-d2d64ce8-920d-4d86-a0b1-386040af3e13'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:03.895164 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-d2d64ce8-920d-4d86-a0b1-386040af3e13 admin admin] The token cache is being invalidate because project 8afcd4d11183401e9e95ed399d2835c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:44:03.895458 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-d2d64ce8-920d-4d86-a0b1-386040af3e13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:03.896121 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-d2d64ce8-920d-4d86-a0b1-386040af3e13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:03.897638 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 941/1880] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:44:03 2022] DELETE /identity/v3/projects/8afcd4d11183401e9e95ed399d2835c3 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:44:03.901677 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0c2cf-24d3-442b-bfac-71709a9f8757 service neutron] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:03.911428 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0c2cf-24d3-442b-bfac-71709a9f8757 service neutron] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:03.941238 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:44:03.971307 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57025c98-d7b1-420e-89a8-1cf06b8c7c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:03.972083 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57025c98-d7b1-420e-89a8-1cf06b8c7c08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:03.972464 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57025c98-d7b1-420e-89a8-1cf06b8c7c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:03.972838 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57025c98-d7b1-420e-89a8-1cf06b8c7c08 admin admin] PATH_INFO: `/v3/users/80d329290b7344959d6eff6df9f263ce` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:03.974204 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57025c98-d7b1-420e-89a8-1cf06b8c7c08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80d329290b7344959d6eff6df9f263ce)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:03.976325 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57025c98-d7b1-420e-89a8-1cf06b8c7c08 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:04.026535 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-57025c98-d7b1-420e-89a8-1cf06b8c7c08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80d329290b7344959d6eff6df9f263ce'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:04.027186 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-57025c98-d7b1-420e-89a8-1cf06b8c7c08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80d329290b7344959d6eff6df9f263ce'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:04.027791 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-57025c98-d7b1-420e-89a8-1cf06b8c7c08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80d329290b7344959d6eff6df9f263ce'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:04.036585 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-57025c98-d7b1-420e-89a8-1cf06b8c7c08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80d329290b7344959d6eff6df9f263ce'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:04.044562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-57025c98-d7b1-420e-89a8-1cf06b8c7c08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80d329290b7344959d6eff6df9f263ce'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:04.064164 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 940/1881] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:44:03 2022] DELETE /identity/v3/users/80d329290b7344959d6eff6df9f263ce => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:44:04.427548 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d64ce8-920d-4d86-a0b1-386040af3e13 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:04.429070 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-d2d64ce8-920d-4d86-a0b1-386040af3e13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:04.447260 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d64ce8-920d-4d86-a0b1-386040af3e13 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:04.487738 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:44:04.509992 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d3f94-df2c-4bda-ad55-e68d601762a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:04.511230 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d3f94-df2c-4bda-ad55-e68d601762a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:04.511518 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d3f94-df2c-4bda-ad55-e68d601762a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:04.511787 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d3f94-df2c-4bda-ad55-e68d601762a3 admin admin] PATH_INFO: `/v3/projects/befa66a7a70f4c93b35a52ba1ab9082c` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:04.512942 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d3f94-df2c-4bda-ad55-e68d601762a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=befa66a7a70f4c93b35a52ba1ab9082c)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:04.514859 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d3f94-df2c-4bda-ad55-e68d601762a3 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:04.563306 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-bd8d3f94-df2c-4bda-ad55-e68d601762a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'befa66a7a70f4c93b35a52ba1ab9082c', 'request_id': 'req-bd8d3f94-df2c-4bda-ad55-e68d601762a3'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:04.569141 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-bd8d3f94-df2c-4bda-ad55-e68d601762a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'befa66a7a70f4c93b35a52ba1ab9082c', 'request_id': 'req-bd8d3f94-df2c-4bda-ad55-e68d601762a3'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:04.577242 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-bd8d3f94-df2c-4bda-ad55-e68d601762a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'befa66a7a70f4c93b35a52ba1ab9082c', 'request_id': 'req-bd8d3f94-df2c-4bda-ad55-e68d601762a3'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:04.589306 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-bd8d3f94-df2c-4bda-ad55-e68d601762a3 admin admin] The token cache is being invalidate because project befa66a7a70f4c93b35a52ba1ab9082c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:44:04.589933 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-bd8d3f94-df2c-4bda-ad55-e68d601762a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:04.590712 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-bd8d3f94-df2c-4bda-ad55-e68d601762a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:04.592712 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 942/1882] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:44:04 2022] DELETE /identity/v3/projects/befa66a7a70f4c93b35a52ba1ab9082c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:44:04.598358 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0010a8d-df1b-482f-9c02-06a85ed65773 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:04.598856 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0010a8d-df1b-482f-9c02-06a85ed65773 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:04.598985 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0010a8d-df1b-482f-9c02-06a85ed65773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:04.619266 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-f0010a8d-df1b-482f-9c02-06a85ed65773 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:44:04.621057 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-f0010a8d-df1b-482f-9c02-06a85ed65773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:04.677590 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 941/1883] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:44:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:44:04.686870 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d138389-642e-4b8b-9c6a-d8b3f6704e4c None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:04.687154 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d138389-642e-4b8b-9c6a-d8b3f6704e4c None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:04.687427 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d138389-642e-4b8b-9c6a-d8b3f6704e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:04.706272 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-1d138389-642e-4b8b-9c6a-d8b3f6704e4c None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:44:04.707461 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-1d138389-642e-4b8b-9c6a-d8b3f6704e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:04.715150 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 943/1884] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:44:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:44:04.719361 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0010a8d-df1b-482f-9c02-06a85ed65773 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:04.729521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0010a8d-df1b-482f-9c02-06a85ed65773 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:04.734338 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8521b3c6-4f78-4bad-a1d7-cef341791230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:04.734939 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521b3c6-4f78-4bad-a1d7-cef341791230 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:04.735825 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521b3c6-4f78-4bad-a1d7-cef341791230 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:04.735825 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521b3c6-4f78-4bad-a1d7-cef341791230 admin admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:04.742323 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8521b3c6-4f78-4bad-a1d7-cef341791230 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:44:04.742587 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8521b3c6-4f78-4bad-a1d7-cef341791230 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:04.744235 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8521b3c6-4f78-4bad-a1d7-cef341791230 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:04.759599 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 942/1885] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:44:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:44:04.761381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d138389-642e-4b8b-9c6a-d8b3f6704e4c None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:04.770605 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d138389-642e-4b8b-9c6a-d8b3f6704e4c None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:04.775241 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af339ceb-7b24-4710-af7b-312fb5d9a658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:04.775835 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af339ceb-7b24-4710-af7b-312fb5d9a658 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:04.776142 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af339ceb-7b24-4710-af7b-312fb5d9a658 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:04.776411 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af339ceb-7b24-4710-af7b-312fb5d9a658 admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:04.777090 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af339ceb-7b24-4710-af7b-312fb5d9a658 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:04.779029 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af339ceb-7b24-4710-af7b-312fb5d9a658 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:04.808581 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 944/1886] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:44:04 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:44:04.812843 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8521b3c6-4f78-4bad-a1d7-cef341791230 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:04.822935 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8521b3c6-4f78-4bad-a1d7-cef341791230 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:04.855022 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:44:04.880377 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b8f7a6-2247-470f-9612-ddcb9aa190e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:04.880975 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8f7a6-2247-470f-9612-ddcb9aa190e6 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:04.881251 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8f7a6-2247-470f-9612-ddcb9aa190e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:04.881522 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8f7a6-2247-470f-9612-ddcb9aa190e6 admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:04.882144 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b8f7a6-2247-470f-9612-ddcb9aa190e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:04.884771 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b8f7a6-2247-470f-9612-ddcb9aa190e6 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:04.910273 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 943/1887] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:44:04 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:44:04.912179 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af339ceb-7b24-4710-af7b-312fb5d9a658 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:04.923362 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af339ceb-7b24-4710-af7b-312fb5d9a658 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:04.927906 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44e8e18-52e8-466a-9352-c20f8ce5720b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:04.928530 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e8e18-52e8-466a-9352-c20f8ce5720b admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:04.928811 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e8e18-52e8-466a-9352-c20f8ce5720b admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:04.929083 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e8e18-52e8-466a-9352-c20f8ce5720b admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:04.929725 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44e8e18-52e8-466a-9352-c20f8ce5720b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:04.931338 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44e8e18-52e8-466a-9352-c20f8ce5720b admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:04.941287 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 945/1888] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:44:04.944988 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b8f7a6-2247-470f-9612-ddcb9aa190e6 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:04.955209 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b8f7a6-2247-470f-9612-ddcb9aa190e6 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:04.959889 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd1aeb-568b-4ece-9ed5-40e03fcba008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:04.960517 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd1aeb-568b-4ece-9ed5-40e03fcba008 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:04.960796 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd1aeb-568b-4ece-9ed5-40e03fcba008 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:04.961066 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd1aeb-568b-4ece-9ed5-40e03fcba008 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:04.961771 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbd1aeb-568b-4ece-9ed5-40e03fcba008 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:04.963806 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbd1aeb-568b-4ece-9ed5-40e03fcba008 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:04.974766 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 944/1889] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:44:04.978753 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44e8e18-52e8-466a-9352-c20f8ce5720b admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:04.990420 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44e8e18-52e8-466a-9352-c20f8ce5720b admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:04.997058 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb258d42-f963-4515-815f-c9cdeba5dc2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:04.997687 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb258d42-f963-4515-815f-c9cdeba5dc2f admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:04.998134 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb258d42-f963-4515-815f-c9cdeba5dc2f admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:04.999928 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb258d42-f963-4515-815f-c9cdeba5dc2f admin admin] PATH_INFO: `/v3/projects/135edfbc128148279cdbf0eb10ad05a2/users/ee47903c6ad741bcaa371ae414a793b9/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:05.015648 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb258d42-f963-4515-815f-c9cdeba5dc2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=135edfbc128148279cdbf0eb10ad05a2, user_id=ee47903c6ad741bcaa371ae414a793b9, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:05.018378 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb258d42-f963-4515-815f-c9cdeba5dc2f admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:05.031302 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 946/1890] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:44:04 2022] PUT /identity/v3/projects/135edfbc128148279cdbf0eb10ad05a2/users/ee47903c6ad741bcaa371ae414a793b9/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:44:05.036569 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99419be0-ba7e-4b41-9402-8acea759f89d None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:05.036857 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99419be0-ba7e-4b41-9402-8acea759f89d None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:05.037133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99419be0-ba7e-4b41-9402-8acea759f89d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:05.055342 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-99419be0-ba7e-4b41-9402-8acea759f89d None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:44:05.056577 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-99419be0-ba7e-4b41-9402-8acea759f89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:05.113788 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 945/1891] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:44:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:44:05.123798 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb258d42-f963-4515-815f-c9cdeba5dc2f admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:05.125617 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-eb258d42-f963-4515-815f-c9cdeba5dc2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:05.184042 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb258d42-f963-4515-815f-c9cdeba5dc2f admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:05.187766 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647c3ff-7606-45e7-a5ff-aa95623ddfb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:05.188404 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647c3ff-7606-45e7-a5ff-aa95623ddfb3 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:05.188683 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647c3ff-7606-45e7-a5ff-aa95623ddfb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:05.188952 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647c3ff-7606-45e7-a5ff-aa95623ddfb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:05.199081 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b647c3ff-7606-45e7-a5ff-aa95623ddfb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:05.200560 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b647c3ff-7606-45e7-a5ff-aa95623ddfb3 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:05.210947 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 947/1892] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:44:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:44:06.000931 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8708a8-17ca-4071-b41f-050d687b91f3 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:06.001473 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8708a8-17ca-4071-b41f-050d687b91f3 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:06.001699 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8708a8-17ca-4071-b41f-050d687b91f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:06.031084 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-9a8708a8-17ca-4071-b41f-050d687b91f3 None None] MFA Rules not processed for user `ee47903c6ad741bcaa371ae414a793b9`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:44:06.032337 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-9a8708a8-17ca-4071-b41f-050d687b91f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:06.105784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647c3ff-7606-45e7-a5ff-aa95623ddfb3 service neutron] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:06.107753 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-b647c3ff-7606-45e7-a5ff-aa95623ddfb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:06.118573 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 946/1893] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:44:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:44:06.120488 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8708a8-17ca-4071-b41f-050d687b91f3 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:06.127749 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647c3ff-7606-45e7-a5ff-aa95623ddfb3 service neutron] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:06.132422 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8708a8-17ca-4071-b41f-050d687b91f3 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:06.133764 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c6e76c-5d26-4cb6-a030-38cbf0de11c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:06.134701 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6e76c-5d26-4cb6-a030-38cbf0de11c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:06.135377 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6e76c-5d26-4cb6-a030-38cbf0de11c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:06.135796 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6e76c-5d26-4cb6-a030-38cbf0de11c4 admin admin] PATH_INFO: `/v3/users/84139191d1ea4798b4f1e48b9a1dbfbf` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:06.137079 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c6e76c-5d26-4cb6-a030-38cbf0de11c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84139191d1ea4798b4f1e48b9a1dbfbf)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:06.137356 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb34d6-0b88-4979-9fe4-5bc0e773654f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:06.138056 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb34d6-0b88-4979-9fe4-5bc0e773654f admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:06.138405 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb34d6-0b88-4979-9fe4-5bc0e773654f admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:06.138747 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb34d6-0b88-4979-9fe4-5bc0e773654f admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:06.139452 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeb34d6-0b88-4979-9fe4-5bc0e773654f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:06.140037 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c6e76c-5d26-4cb6-a030-38cbf0de11c4 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:06.141577 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeb34d6-0b88-4979-9fe4-5bc0e773654f admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:06.143717 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:44:06.175315 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 947/1894] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:44:06 2022] POST /identity/v3/projects => generated 369 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:44:06.178695 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb34d6-0b88-4979-9fe4-5bc0e773654f admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:06.206726 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-31c6e76c-5d26-4cb6-a030-38cbf0de11c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84139191d1ea4798b4f1e48b9a1dbfbf'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:06.207504 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-31c6e76c-5d26-4cb6-a030-38cbf0de11c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84139191d1ea4798b4f1e48b9a1dbfbf'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:06.216203 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:44:06.223829 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-31c6e76c-5d26-4cb6-a030-38cbf0de11c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84139191d1ea4798b4f1e48b9a1dbfbf'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:06.235706 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-31c6e76c-5d26-4cb6-a030-38cbf0de11c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84139191d1ea4798b4f1e48b9a1dbfbf'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:06.236696 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-31c6e76c-5d26-4cb6-a030-38cbf0de11c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84139191d1ea4798b4f1e48b9a1dbfbf'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:06.250070 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 948/1895] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:44:06 2022] DELETE /identity/v3/users/84139191d1ea4798b4f1e48b9a1dbfbf => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:44:06.254353 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f4e2fb-249e-47f0-89d7-011ae2d68993 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:06.254636 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f4e2fb-249e-47f0-89d7-011ae2d68993 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:06.254904 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f4e2fb-249e-47f0-89d7-011ae2d68993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:06.256612 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb34d6-0b88-4979-9fe4-5bc0e773654f admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:06.273671 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-e6f4e2fb-249e-47f0-89d7-011ae2d68993 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:44:06.274860 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-e6f4e2fb-249e-47f0-89d7-011ae2d68993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:06.317903 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3b998-cd32-419d-848d-e1cfe5050cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:06.320156 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3b998-cd32-419d-848d-e1cfe5050cf1 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:06.320525 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3b998-cd32-419d-848d-e1cfe5050cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:06.320874 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3b998-cd32-419d-848d-e1cfe5050cf1 admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:06.321586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d3b998-cd32-419d-848d-e1cfe5050cf1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:06.323618 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d3b998-cd32-419d-848d-e1cfe5050cf1 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:06.342291 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 949/1896] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:44:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:44:06.350783 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f4e2fb-249e-47f0-89d7-011ae2d68993 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:06.352926 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 948/1897] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:44:06 2022] POST /identity/v3/users => generated 291 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:44:06.354422 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-e6f4e2fb-249e-47f0-89d7-011ae2d68993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:06.356202 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3b998-cd32-419d-848d-e1cfe5050cf1 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:06.358083 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-74d3b998-cd32-419d-848d-e1cfe5050cf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:06.370904 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3b998-cd32-419d-848d-e1cfe5050cf1 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:06.375547 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a1b01-6e9e-4068-96ce-56bf6569e35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:06.376179 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a1b01-6e9e-4068-96ce-56bf6569e35f admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:06.376461 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a1b01-6e9e-4068-96ce-56bf6569e35f admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:06.376859 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a1b01-6e9e-4068-96ce-56bf6569e35f admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:06.377507 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57a1b01-6e9e-4068-96ce-56bf6569e35f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:06.379107 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57a1b01-6e9e-4068-96ce-56bf6569e35f admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:06.393642 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 949/1898] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:44:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:44:06.395286 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a1b01-6e9e-4068-96ce-56bf6569e35f admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:06.405835 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a1b01-6e9e-4068-96ce-56bf6569e35f admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:06.410973 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab016374-7f58-4a08-80ab-fd6f4c904b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:06.411561 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab016374-7f58-4a08-80ab-fd6f4c904b4a admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:06.411844 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab016374-7f58-4a08-80ab-fd6f4c904b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:06.412267 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab016374-7f58-4a08-80ab-fd6f4c904b4a admin admin] PATH_INFO: `/v3/projects/7f2944f3ccf74c45a9ca6105199ab1cd/users/ac26cb13b643491cb7206ac74a7a5914/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:06.428023 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab016374-7f58-4a08-80ab-fd6f4c904b4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f2944f3ccf74c45a9ca6105199ab1cd, user_id=ac26cb13b643491cb7206ac74a7a5914, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:06.430861 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab016374-7f58-4a08-80ab-fd6f4c904b4a admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:06.431738 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f4e2fb-249e-47f0-89d7-011ae2d68993 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:06.436442 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0a0104-55f0-4fdc-ab7f-c8a7e316c167 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:06.437120 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a0104-55f0-4fdc-ab7f-c8a7e316c167 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:06.437504 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a0104-55f0-4fdc-ab7f-c8a7e316c167 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:06.437741 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a0104-55f0-4fdc-ab7f-c8a7e316c167 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:06.454651 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 950/1899] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:44:06 2022] PUT /identity/v3/projects/7f2944f3ccf74c45a9ca6105199ab1cd/users/ac26cb13b643491cb7206ac74a7a5914/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:44:06.456071 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0a0104-55f0-4fdc-ab7f-c8a7e316c167 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:06.457735 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0a0104-55f0-4fdc-ab7f-c8a7e316c167 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:06.458672 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab016374-7f58-4a08-80ab-fd6f4c904b4a admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:06.471084 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 950/1900] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:44:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 23:44:06.530854 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab016374-7f58-4a08-80ab-fd6f4c904b4a admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:06.542421 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b9b0ce-fa35-4fa5-bf7c-3d993fa80ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:06.543044 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9b0ce-fa35-4fa5-bf7c-3d993fa80ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:06.543322 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9b0ce-fa35-4fa5-bf7c-3d993fa80ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:06.543589 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9b0ce-fa35-4fa5-bf7c-3d993fa80ff7 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:06.544269 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b9b0ce-fa35-4fa5-bf7c-3d993fa80ff7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:06.545969 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b9b0ce-fa35-4fa5-bf7c-3d993fa80ff7 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:06.560138 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 951/1901] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:44:06 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 23:44:06.561579 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0a0104-55f0-4fdc-ab7f-c8a7e316c167 service neutron] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:06.571274 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0a0104-55f0-4fdc-ab7f-c8a7e316c167 service neutron] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:06.577775 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f791446-da8b-4bd8-8bb0-f15cc62c3356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:06.578639 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f791446-da8b-4bd8-8bb0-f15cc62c3356 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:06.578937 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f791446-da8b-4bd8-8bb0-f15cc62c3356 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:06.579559 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f791446-da8b-4bd8-8bb0-f15cc62c3356 admin admin] PATH_INFO: `/v3/projects/7f2944f3ccf74c45a9ca6105199ab1cd/users/ac26cb13b643491cb7206ac74a7a5914/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:06.581521 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f791446-da8b-4bd8-8bb0-f15cc62c3356 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f2944f3ccf74c45a9ca6105199ab1cd, user_id=ac26cb13b643491cb7206ac74a7a5914, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:06.584447 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f791446-da8b-4bd8-8bb0-f15cc62c3356 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:06.596645 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 951/1902] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:44:06 2022] PUT /identity/v3/projects/7f2944f3ccf74c45a9ca6105199ab1cd/users/ac26cb13b643491cb7206ac74a7a5914/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:44:06.995106 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b9b0ce-fa35-4fa5-bf7c-3d993fa80ff7 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:06.997150 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-d7b9b0ce-fa35-4fa5-bf7c-3d993fa80ff7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:07.011183 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b9b0ce-fa35-4fa5-bf7c-3d993fa80ff7 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:07.055307 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:44:07.101068 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54deda75-9956-4b98-aedd-9872202c000b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:07.102045 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54deda75-9956-4b98-aedd-9872202c000b admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:07.102045 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54deda75-9956-4b98-aedd-9872202c000b admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:07.103623 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54deda75-9956-4b98-aedd-9872202c000b admin admin] PATH_INFO: `/v3/projects/cf76f8b2738f480a92be57313e01e6f0` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:07.104906 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54deda75-9956-4b98-aedd-9872202c000b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf76f8b2738f480a92be57313e01e6f0)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:07.106875 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54deda75-9956-4b98-aedd-9872202c000b admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:07.169584 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-54deda75-9956-4b98-aedd-9872202c000b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf76f8b2738f480a92be57313e01e6f0', 'request_id': 'req-54deda75-9956-4b98-aedd-9872202c000b'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:07.176871 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-54deda75-9956-4b98-aedd-9872202c000b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf76f8b2738f480a92be57313e01e6f0', 'request_id': 'req-54deda75-9956-4b98-aedd-9872202c000b'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:07.187292 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-54deda75-9956-4b98-aedd-9872202c000b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf76f8b2738f480a92be57313e01e6f0', 'request_id': 'req-54deda75-9956-4b98-aedd-9872202c000b'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:07.202463 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-54deda75-9956-4b98-aedd-9872202c000b admin admin] The token cache is being invalidate because project cf76f8b2738f480a92be57313e01e6f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:44:07.203023 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-54deda75-9956-4b98-aedd-9872202c000b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:07.203916 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-54deda75-9956-4b98-aedd-9872202c000b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:07.205901 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 952/1903] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:44:06 2022] DELETE /identity/v3/projects/cf76f8b2738f480a92be57313e01e6f0 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:44:07.210041 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f791446-da8b-4bd8-8bb0-f15cc62c3356 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:07.212363 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-1f791446-da8b-4bd8-8bb0-f15cc62c3356 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:07.227752 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f791446-da8b-4bd8-8bb0-f15cc62c3356 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:07.238912 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:44:07.321777 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4a0a0-ba84-40ae-9a65-9e1827458d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:07.321939 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4a0a0-ba84-40ae-9a65-9e1827458d44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:07.322090 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4a0a0-ba84-40ae-9a65-9e1827458d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:07.322362 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4a0a0-ba84-40ae-9a65-9e1827458d44 admin admin] PATH_INFO: `/v3/users/f30081ccfc5c4a5ba6c8e93863aa00bd` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:07.324121 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb4a0a0-ba84-40ae-9a65-9e1827458d44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f30081ccfc5c4a5ba6c8e93863aa00bd)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:07.326140 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb4a0a0-ba84-40ae-9a65-9e1827458d44 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:07.404095 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-9bb4a0a0-ba84-40ae-9a65-9e1827458d44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f30081ccfc5c4a5ba6c8e93863aa00bd'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:07.404694 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-9bb4a0a0-ba84-40ae-9a65-9e1827458d44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f30081ccfc5c4a5ba6c8e93863aa00bd'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:07.418979 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-9bb4a0a0-ba84-40ae-9a65-9e1827458d44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f30081ccfc5c4a5ba6c8e93863aa00bd'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:07.428830 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-9bb4a0a0-ba84-40ae-9a65-9e1827458d44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f30081ccfc5c4a5ba6c8e93863aa00bd'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:07.429674 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-9bb4a0a0-ba84-40ae-9a65-9e1827458d44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f30081ccfc5c4a5ba6c8e93863aa00bd'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:07.456679 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 952/1904] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:44:07 2022] DELETE /identity/v3/users/f30081ccfc5c4a5ba6c8e93863aa00bd => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:44:07.535346 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40db5d3-c9c8-4221-aea4-5274eff94ea0 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:07.536166 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40db5d3-c9c8-4221-aea4-5274eff94ea0 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:07.536660 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40db5d3-c9c8-4221-aea4-5274eff94ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:07.576567 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-a40db5d3-c9c8-4221-aea4-5274eff94ea0 None None] MFA Rules not processed for user `ac26cb13b643491cb7206ac74a7a5914`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:44:07.577972 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-a40db5d3-c9c8-4221-aea4-5274eff94ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:07.674652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 953/1905] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:44:07 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:44:07.687758 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4a0a0-ba84-40ae-9a65-9e1827458d44 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:07.689410 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-9bb4a0a0-ba84-40ae-9a65-9e1827458d44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:07.766246 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4a0a0-ba84-40ae-9a65-9e1827458d44 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:07.770111 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a01f0e-13c9-46ea-acde-f3ea98457192 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:07.770722 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a01f0e-13c9-46ea-acde-f3ea98457192 service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:07.771005 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a01f0e-13c9-46ea-acde-f3ea98457192 service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:07.771275 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a01f0e-13c9-46ea-acde-f3ea98457192 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:07.773013 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-18a01f0e-13c9-46ea-acde-f3ea98457192 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:07.821029 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:44:07.830032 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40db5d3-c9c8-4221-aea4-5274eff94ea0 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:07.831756 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-a40db5d3-c9c8-4221-aea4-5274eff94ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:07.837799 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a01f0e-13c9-46ea-acde-f3ea98457192 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:07.839321 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a01f0e-13c9-46ea-acde-f3ea98457192 service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:07.850880 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 953/1906] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:44:07 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:44:07.880531 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:44:07.912104 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40db5d3-c9c8-4221-aea4-5274eff94ea0 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:07.918633 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d2d2a-8963-4f25-86a5-1bb7073d5848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:07.919278 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d2d2a-8963-4f25-86a5-1bb7073d5848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:07.919556 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d2d2a-8963-4f25-86a5-1bb7073d5848 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:07.919830 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d2d2a-8963-4f25-86a5-1bb7073d5848 admin admin] PATH_INFO: `/v3/projects/e7446ac4f108444984a9702688b910dc` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:07.920989 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9d2d2a-8963-4f25-86a5-1bb7073d5848 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7446ac4f108444984a9702688b910dc)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:07.922915 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9d2d2a-8963-4f25-86a5-1bb7073d5848 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:07.989789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-1f9d2d2a-8963-4f25-86a5-1bb7073d5848 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7446ac4f108444984a9702688b910dc', 'request_id': 'req-1f9d2d2a-8963-4f25-86a5-1bb7073d5848'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:07.999917 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-1f9d2d2a-8963-4f25-86a5-1bb7073d5848 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7446ac4f108444984a9702688b910dc', 'request_id': 'req-1f9d2d2a-8963-4f25-86a5-1bb7073d5848'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:08.012439 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-1f9d2d2a-8963-4f25-86a5-1bb7073d5848 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7446ac4f108444984a9702688b910dc', 'request_id': 'req-1f9d2d2a-8963-4f25-86a5-1bb7073d5848'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:08.026214 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-1f9d2d2a-8963-4f25-86a5-1bb7073d5848 admin admin] The token cache is being invalidate because project e7446ac4f108444984a9702688b910dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:44:08.026586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-1f9d2d2a-8963-4f25-86a5-1bb7073d5848 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:08.027404 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-1f9d2d2a-8963-4f25-86a5-1bb7073d5848 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:08.029535 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 954/1907] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:44:07 2022] DELETE /identity/v3/projects/e7446ac4f108444984a9702688b910dc => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:44:08.034200 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a01f0e-13c9-46ea-acde-f3ea98457192 service cinder] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:08.058250 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a01f0e-13c9-46ea-acde-f3ea98457192 service cinder] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:08.137418 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d124e-ab7d-4e05-9233-6f113367c6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:08.138749 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d124e-ab7d-4e05-9233-6f113367c6fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:08.139236 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d124e-ab7d-4e05-9233-6f113367c6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:08.139710 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d124e-ab7d-4e05-9233-6f113367c6fb admin admin] PATH_INFO: `/v3/users/5267e5b3d0ae4a57985e72eb27b0ad57` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:08.141940 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d124e-ab7d-4e05-9233-6f113367c6fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=5267e5b3d0ae4a57985e72eb27b0ad57)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:08.145062 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d124e-ab7d-4e05-9233-6f113367c6fb admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:08.222992 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-f43d124e-ab7d-4e05-9233-6f113367c6fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5267e5b3d0ae4a57985e72eb27b0ad57'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:08.225234 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-f43d124e-ab7d-4e05-9233-6f113367c6fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5267e5b3d0ae4a57985e72eb27b0ad57'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:08.241527 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-f43d124e-ab7d-4e05-9233-6f113367c6fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5267e5b3d0ae4a57985e72eb27b0ad57'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:08.262227 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-f43d124e-ab7d-4e05-9233-6f113367c6fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5267e5b3d0ae4a57985e72eb27b0ad57'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:08.263185 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-f43d124e-ab7d-4e05-9233-6f113367c6fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5267e5b3d0ae4a57985e72eb27b0ad57'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:08.281688 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 954/1908] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:44:08 2022] DELETE /identity/v3/users/5267e5b3d0ae4a57985e72eb27b0ad57 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:44:08.555108 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d2d2a-8963-4f25-86a5-1bb7073d5848 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:08.556573 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-1f9d2d2a-8963-4f25-86a5-1bb7073d5848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:08.570527 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d2d2a-8963-4f25-86a5-1bb7073d5848 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:08.626241 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbbd179-3918-4b25-acdb-ddf0c0277d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:08.626873 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbbd179-3918-4b25-acdb-ddf0c0277d3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:08.627173 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbbd179-3918-4b25-acdb-ddf0c0277d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:08.627436 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbbd179-3918-4b25-acdb-ddf0c0277d3a admin admin] PATH_INFO: `/v3/projects/40ddea386ca64c8c8f15041f994d75bb` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:08.628574 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbbd179-3918-4b25-acdb-ddf0c0277d3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=40ddea386ca64c8c8f15041f994d75bb)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:08.630515 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbbd179-3918-4b25-acdb-ddf0c0277d3a admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:08.675456 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-7dbbd179-3918-4b25-acdb-ddf0c0277d3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40ddea386ca64c8c8f15041f994d75bb', 'request_id': 'req-7dbbd179-3918-4b25-acdb-ddf0c0277d3a'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:08.680719 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-7dbbd179-3918-4b25-acdb-ddf0c0277d3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40ddea386ca64c8c8f15041f994d75bb', 'request_id': 'req-7dbbd179-3918-4b25-acdb-ddf0c0277d3a'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:08.687759 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-7dbbd179-3918-4b25-acdb-ddf0c0277d3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40ddea386ca64c8c8f15041f994d75bb', 'request_id': 'req-7dbbd179-3918-4b25-acdb-ddf0c0277d3a'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:08.694421 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:44:08.700411 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-7dbbd179-3918-4b25-acdb-ddf0c0277d3a admin admin] The token cache is being invalidate because project 40ddea386ca64c8c8f15041f994d75bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:44:08.700919 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-7dbbd179-3918-4b25-acdb-ddf0c0277d3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:08.701981 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-7dbbd179-3918-4b25-acdb-ddf0c0277d3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:08.703643 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 955/1909] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:44:08 2022] DELETE /identity/v3/projects/40ddea386ca64c8c8f15041f994d75bb => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:44:08.709440 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b1e20-3d37-408f-9344-5b370e4f4616 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:08.709726 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b1e20-3d37-408f-9344-5b370e4f4616 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:08.709997 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b1e20-3d37-408f-9344-5b370e4f4616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:08.729907 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-6f4b1e20-3d37-408f-9344-5b370e4f4616 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:44:08.731090 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-6f4b1e20-3d37-408f-9344-5b370e4f4616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:08.792558 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 955/1910] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:44:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:44:08.800278 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fc995-fef2-477d-8a31-648886ed882e None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:08.800564 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fc995-fef2-477d-8a31-648886ed882e None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:08.800842 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fc995-fef2-477d-8a31-648886ed882e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:08.821463 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-601fc995-fef2-477d-8a31-648886ed882e None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:44:08.823336 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-601fc995-fef2-477d-8a31-648886ed882e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:08.831363 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 956/1911] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:44:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:44:08.835582 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4b1e20-3d37-408f-9344-5b370e4f4616 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:08.842084 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:44:08.846796 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4b1e20-3d37-408f-9344-5b370e4f4616 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:08.852661 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964109de-f3a3-4a1c-b4ab-1d103faa20e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:08.853282 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964109de-f3a3-4a1c-b4ab-1d103faa20e0 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:08.853777 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964109de-f3a3-4a1c-b4ab-1d103faa20e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:08.854342 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964109de-f3a3-4a1c-b4ab-1d103faa20e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:08.855024 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964109de-f3a3-4a1c-b4ab-1d103faa20e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:44:08.855327 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964109de-f3a3-4a1c-b4ab-1d103faa20e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:08.856991 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964109de-f3a3-4a1c-b4ab-1d103faa20e0 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:08.869974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 956/1912] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:44:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:44:08.874282 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601fc995-fef2-477d-8a31-648886ed882e None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:08.883633 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601fc995-fef2-477d-8a31-648886ed882e None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:08.888841 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e264d5-1b3d-48f9-bfc0-7f2d8bedff89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:08.889446 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e264d5-1b3d-48f9-bfc0-7f2d8bedff89 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:08.889723 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e264d5-1b3d-48f9-bfc0-7f2d8bedff89 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:08.889994 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e264d5-1b3d-48f9-bfc0-7f2d8bedff89 admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:08.890629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e264d5-1b3d-48f9-bfc0-7f2d8bedff89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:08.892564 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e264d5-1b3d-48f9-bfc0-7f2d8bedff89 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:08.920882 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 957/1913] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:44:08 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:44:08.925040 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964109de-f3a3-4a1c-b4ab-1d103faa20e0 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:08.936947 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964109de-f3a3-4a1c-b4ab-1d103faa20e0 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:08.995583 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084cddd-719e-4069-b759-4ae3e741836b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:08.996240 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084cddd-719e-4069-b759-4ae3e741836b admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:08.996521 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084cddd-719e-4069-b759-4ae3e741836b admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:08.996828 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084cddd-719e-4069-b759-4ae3e741836b admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:08.997456 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1084cddd-719e-4069-b759-4ae3e741836b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:08.999396 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1084cddd-719e-4069-b759-4ae3e741836b admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:09.021586 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 957/1914] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:44:08 2022] POST /identity/v3/users => generated 282 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:44:09.025685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e264d5-1b3d-48f9-bfc0-7f2d8bedff89 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:09.036447 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e264d5-1b3d-48f9-bfc0-7f2d8bedff89 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:09.041036 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5123b39d-f843-4944-afa8-34e67f785e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:09.041664 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123b39d-f843-4944-afa8-34e67f785e41 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:09.041942 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123b39d-f843-4944-afa8-34e67f785e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:09.042213 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123b39d-f843-4944-afa8-34e67f785e41 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:09.042858 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5123b39d-f843-4944-afa8-34e67f785e41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:09.044521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5123b39d-f843-4944-afa8-34e67f785e41 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:09.054941 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 958/1915] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:44:09.059296 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084cddd-719e-4069-b759-4ae3e741836b admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:09.069438 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084cddd-719e-4069-b759-4ae3e741836b admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:09.074623 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401fbfbf-c5b8-4cb0-bf86-81fce81d44b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:09.075218 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fbfbf-c5b8-4cb0-bf86-81fce81d44b5 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:09.075900 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fbfbf-c5b8-4cb0-bf86-81fce81d44b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:09.075900 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fbfbf-c5b8-4cb0-bf86-81fce81d44b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:09.076418 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401fbfbf-c5b8-4cb0-bf86-81fce81d44b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:09.078097 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401fbfbf-c5b8-4cb0-bf86-81fce81d44b5 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:09.088612 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 958/1916] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:44:09.092709 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5123b39d-f843-4944-afa8-34e67f785e41 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:09.102825 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5123b39d-f843-4944-afa8-34e67f785e41 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:09.107937 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa121c-2403-4f99-a7bd-e4098efa38be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:09.108627 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa121c-2403-4f99-a7bd-e4098efa38be admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:09.108908 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa121c-2403-4f99-a7bd-e4098efa38be admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:09.109185 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa121c-2403-4f99-a7bd-e4098efa38be admin admin] PATH_INFO: `/v3/projects/fc6705fbfa7342c781f12fede37322b0/users/592d6d184a9b421ea2f2d20232ddea14/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:09.124831 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aa121c-2403-4f99-a7bd-e4098efa38be admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc6705fbfa7342c781f12fede37322b0, user_id=592d6d184a9b421ea2f2d20232ddea14, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:09.127552 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aa121c-2403-4f99-a7bd-e4098efa38be admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:09.148118 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 959/1917] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:44:09 2022] PUT /identity/v3/projects/fc6705fbfa7342c781f12fede37322b0/users/592d6d184a9b421ea2f2d20232ddea14/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:44:09.157569 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f42b4f-07fb-4c6b-be43-20b62bae8f57 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:09.157854 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f42b4f-07fb-4c6b-be43-20b62bae8f57 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:09.158130 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f42b4f-07fb-4c6b-be43-20b62bae8f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:09.188145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-a9f42b4f-07fb-4c6b-be43-20b62bae8f57 None None] MFA Rules not processed for user `592d6d184a9b421ea2f2d20232ddea14`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:44:09.189650 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-a9f42b4f-07fb-4c6b-be43-20b62bae8f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:09.283504 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 959/1918] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:44:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:44:09.310731 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa121c-2403-4f99-a7bd-e4098efa38be admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:09.312642 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-42aa121c-2403-4f99-a7bd-e4098efa38be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:09.375233 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa121c-2403-4f99-a7bd-e4098efa38be admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:09.379353 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80180f53-778f-4fb8-9491-d059b2ce46a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:09.379986 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80180f53-778f-4fb8-9491-d059b2ce46a6 service cinder] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:09.380323 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80180f53-778f-4fb8-9491-d059b2ce46a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:09.380539 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80180f53-778f-4fb8-9491-d059b2ce46a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:09.390588 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80180f53-778f-4fb8-9491-d059b2ce46a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:09.392104 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80180f53-778f-4fb8-9491-d059b2ce46a6 service cinder] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:09.402562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 960/1919] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:44:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:44:13.580288 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f42b4f-07fb-4c6b-be43-20b62bae8f57 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:13.590318 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f42b4f-07fb-4c6b-be43-20b62bae8f57 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:13.594016 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c96ec4-c2c2-485a-ae08-90357a9842d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:13.594609 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c96ec4-c2c2-485a-ae08-90357a9842d4 service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:13.594889 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c96ec4-c2c2-485a-ae08-90357a9842d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:13.595173 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c96ec4-c2c2-485a-ae08-90357a9842d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:13.597413 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-07c96ec4-c2c2-485a-ae08-90357a9842d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:13.669400 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:44:13.673454 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c96ec4-c2c2-485a-ae08-90357a9842d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:13.675229 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c96ec4-c2c2-485a-ae08-90357a9842d4 service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:13.687426 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 960/1920] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:44:13 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:44:14.466861 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6cdfeb-c2c5-49d8-bb92-fe16e41af051 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:14.467375 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6cdfeb-c2c5-49d8-bb92-fe16e41af051 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:14.467466 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6cdfeb-c2c5-49d8-bb92-fe16e41af051 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:14.470086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 961/1921] 10.222.0.26 () {54 vars in 921 bytes} [Wed Aug 17 23:44:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:44:14.480896 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d364705-8559-463e-94ee-11fa421c379e None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:14.481190 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d364705-8559-463e-94ee-11fa421c379e None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:14.481459 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d364705-8559-463e-94ee-11fa421c379e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:14.509098 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-4d364705-8559-463e-94ee-11fa421c379e None None] MFA Rules not processed for user `ec1dcbe43b0749c3bcc4af9100d771d1`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:44:14.511155 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-4d364705-8559-463e-94ee-11fa421c379e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:14.647667 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 961/1922] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Aug 17 23:44:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:44:14.740080 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6cdfeb-c2c5-49d8-bb92-fe16e41af051 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:14.741913 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-6f6cdfeb-c2c5-49d8-bb92-fe16e41af051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:14.761622 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6cdfeb-c2c5-49d8-bb92-fe16e41af051 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:14.766968 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527615a7-20cc-4f2f-bd88-23a4ec77aadf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:14.767569 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527615a7-20cc-4f2f-bd88-23a4ec77aadf service nova] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:14.767841 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527615a7-20cc-4f2f-bd88-23a4ec77aadf service nova] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:14.768298 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527615a7-20cc-4f2f-bd88-23a4ec77aadf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:14.777488 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527615a7-20cc-4f2f-bd88-23a4ec77aadf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:14.779420 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527615a7-20cc-4f2f-bd88-23a4ec77aadf service nova] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:14.791323 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 962/1923] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:44:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 23:44:24.636696 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=34 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:44:24.787237 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=60,keystone:INSERT=2 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:44:51.915728 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d364705-8559-463e-94ee-11fa421c379e None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:51.917195 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-4d364705-8559-463e-94ee-11fa421c379e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:51.982746 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d364705-8559-463e-94ee-11fa421c379e None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:51.987428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f61409-cb99-4ff4-9c07-fac70bc6376a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:51.988164 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f61409-cb99-4ff4-9c07-fac70bc6376a admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:51.988448 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f61409-cb99-4ff4-9c07-fac70bc6376a admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:51.988723 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f61409-cb99-4ff4-9c07-fac70bc6376a admin admin] PATH_INFO: `/v3/users/245604b5ad2244d6b0662fbf2cb02f5f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:51.989871 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f61409-cb99-4ff4-9c07-fac70bc6376a admin admin] RBAC: Authorizing `identity:delete_user(user_id=245604b5ad2244d6b0662fbf2cb02f5f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:51.992089 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f61409-cb99-4ff4-9c07-fac70bc6376a admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:52.041118 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-e4f61409-cb99-4ff4-9c07-fac70bc6376a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '245604b5ad2244d6b0662fbf2cb02f5f'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:52.041778 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-e4f61409-cb99-4ff4-9c07-fac70bc6376a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '245604b5ad2244d6b0662fbf2cb02f5f'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:52.061117 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-e4f61409-cb99-4ff4-9c07-fac70bc6376a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '245604b5ad2244d6b0662fbf2cb02f5f'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:52.070851 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-e4f61409-cb99-4ff4-9c07-fac70bc6376a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '245604b5ad2244d6b0662fbf2cb02f5f'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:52.071367 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-e4f61409-cb99-4ff4-9c07-fac70bc6376a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '245604b5ad2244d6b0662fbf2cb02f5f'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:52.092795 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 962/1924] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:44:51 2022] DELETE /identity/v3/users/245604b5ad2244d6b0662fbf2cb02f5f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:44:52.093985 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3a6e0a-ea40-4b56-aae5-486e94bbde75 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:52.094508 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3a6e0a-ea40-4b56-aae5-486e94bbde75 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:52.094600 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3a6e0a-ea40-4b56-aae5-486e94bbde75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:52.114933 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-ed3a6e0a-ea40-4b56-aae5-486e94bbde75 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:44:52.116413 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-ed3a6e0a-ea40-4b56-aae5-486e94bbde75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:52.238995 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 963/1925] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:44:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:44:52.318929 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f61409-cb99-4ff4-9c07-fac70bc6376a admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:52.321181 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-e4f61409-cb99-4ff4-9c07-fac70bc6376a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:52.430895 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f61409-cb99-4ff4-9c07-fac70bc6376a admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:52.440478 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d982a34-4b63-42f5-bec3-5527eaa8724d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:52.441220 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d982a34-4b63-42f5-bec3-5527eaa8724d service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:52.441405 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d982a34-4b63-42f5-bec3-5527eaa8724d service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:52.441739 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d982a34-4b63-42f5-bec3-5527eaa8724d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:52.454435 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d982a34-4b63-42f5-bec3-5527eaa8724d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:52.456521 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d982a34-4b63-42f5-bec3-5527eaa8724d service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:52.473442 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 963/1926] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:44:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:44:53.236212 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3a6e0a-ea40-4b56-aae5-486e94bbde75 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:53.238380 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-ed3a6e0a-ea40-4b56-aae5-486e94bbde75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:53.251768 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3a6e0a-ea40-4b56-aae5-486e94bbde75 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:53.256763 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc985d54-7003-4905-baeb-8dd227c17453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:53.258463 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc985d54-7003-4905-baeb-8dd227c17453 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:53.258897 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc985d54-7003-4905-baeb-8dd227c17453 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:53.260236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc985d54-7003-4905-baeb-8dd227c17453 admin admin] PATH_INFO: `/v3/projects/506962f3cc994fedb5f2f3fbfc4efdb2` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:53.262561 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc985d54-7003-4905-baeb-8dd227c17453 admin admin] RBAC: Authorizing `identity:delete_project(project_id=506962f3cc994fedb5f2f3fbfc4efdb2)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:53.264982 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc985d54-7003-4905-baeb-8dd227c17453 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:53.314802 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-cc985d54-7003-4905-baeb-8dd227c17453 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '506962f3cc994fedb5f2f3fbfc4efdb2', 'request_id': 'req-cc985d54-7003-4905-baeb-8dd227c17453'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:53.320708 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-cc985d54-7003-4905-baeb-8dd227c17453 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '506962f3cc994fedb5f2f3fbfc4efdb2', 'request_id': 'req-cc985d54-7003-4905-baeb-8dd227c17453'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:53.329415 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-cc985d54-7003-4905-baeb-8dd227c17453 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '506962f3cc994fedb5f2f3fbfc4efdb2', 'request_id': 'req-cc985d54-7003-4905-baeb-8dd227c17453'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:53.341155 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-cc985d54-7003-4905-baeb-8dd227c17453 admin admin] The token cache is being invalidate because project 506962f3cc994fedb5f2f3fbfc4efdb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:44:53.341428 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-cc985d54-7003-4905-baeb-8dd227c17453 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:53.342074 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-cc985d54-7003-4905-baeb-8dd227c17453 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:53.343575 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 964/1927] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:44:53 2022] DELETE /identity/v3/projects/506962f3cc994fedb5f2f3fbfc4efdb2 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:44:53.347589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d982a34-4b63-42f5-bec3-5527eaa8724d service neutron] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:53.357387 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d982a34-4b63-42f5-bec3-5527eaa8724d service neutron] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:53.379065 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:44:53.429051 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e7da49-f68e-45e7-a11f-7beb0cbe89a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:53.429947 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e7da49-f68e-45e7-a11f-7beb0cbe89a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:53.430304 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e7da49-f68e-45e7-a11f-7beb0cbe89a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:53.430573 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e7da49-f68e-45e7-a11f-7beb0cbe89a2 admin admin] PATH_INFO: `/v3/users/f4187ce485d54c159bfdff2fe384a138` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:53.431701 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e7da49-f68e-45e7-a11f-7beb0cbe89a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4187ce485d54c159bfdff2fe384a138)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:53.434387 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e7da49-f68e-45e7-a11f-7beb0cbe89a2 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:53.497426 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-a8e7da49-f68e-45e7-a11f-7beb0cbe89a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4187ce485d54c159bfdff2fe384a138'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:53.498097 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-a8e7da49-f68e-45e7-a11f-7beb0cbe89a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4187ce485d54c159bfdff2fe384a138'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:53.510799 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-a8e7da49-f68e-45e7-a11f-7beb0cbe89a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4187ce485d54c159bfdff2fe384a138'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:53.519806 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-a8e7da49-f68e-45e7-a11f-7beb0cbe89a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4187ce485d54c159bfdff2fe384a138'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:53.520460 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-a8e7da49-f68e-45e7-a11f-7beb0cbe89a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4187ce485d54c159bfdff2fe384a138'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:53.532583 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 964/1928] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:44:53 2022] DELETE /identity/v3/users/f4187ce485d54c159bfdff2fe384a138 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:44:53.918238 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc985d54-7003-4905-baeb-8dd227c17453 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:53.919823 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-cc985d54-7003-4905-baeb-8dd227c17453 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:53.942078 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc985d54-7003-4905-baeb-8dd227c17453 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:54.033982 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3050d0f6-748f-45ff-ab61-1b74fd3ee4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:54.034918 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050d0f6-748f-45ff-ab61-1b74fd3ee4a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:54.035678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050d0f6-748f-45ff-ab61-1b74fd3ee4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:54.036101 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050d0f6-748f-45ff-ab61-1b74fd3ee4a8 admin admin] PATH_INFO: `/v3/projects/3f07a425cb3542ff8d9e212afb778f58` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:54.038228 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3050d0f6-748f-45ff-ab61-1b74fd3ee4a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f07a425cb3542ff8d9e212afb778f58)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:54.042191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3050d0f6-748f-45ff-ab61-1b74fd3ee4a8 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:54.115660 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:44:54.147926 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-3050d0f6-748f-45ff-ab61-1b74fd3ee4a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f07a425cb3542ff8d9e212afb778f58', 'request_id': 'req-3050d0f6-748f-45ff-ab61-1b74fd3ee4a8'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:54.178656 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-3050d0f6-748f-45ff-ab61-1b74fd3ee4a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f07a425cb3542ff8d9e212afb778f58', 'request_id': 'req-3050d0f6-748f-45ff-ab61-1b74fd3ee4a8'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:54.188976 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-3050d0f6-748f-45ff-ab61-1b74fd3ee4a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f07a425cb3542ff8d9e212afb778f58', 'request_id': 'req-3050d0f6-748f-45ff-ab61-1b74fd3ee4a8'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:54.204686 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-3050d0f6-748f-45ff-ab61-1b74fd3ee4a8 admin admin] The token cache is being invalidate because project 3f07a425cb3542ff8d9e212afb778f58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:44:54.205115 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-3050d0f6-748f-45ff-ab61-1b74fd3ee4a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:54.205989 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-3050d0f6-748f-45ff-ab61-1b74fd3ee4a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:54.221367 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 965/1929] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:44:53 2022] DELETE /identity/v3/projects/3f07a425cb3542ff8d9e212afb778f58 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:44:54.222259 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2abbcb5-74ef-420f-97e2-eae5cbc75de5 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:54.222548 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2abbcb5-74ef-420f-97e2-eae5cbc75de5 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:54.231539 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2abbcb5-74ef-420f-97e2-eae5cbc75de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:54.256588 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-b2abbcb5-74ef-420f-97e2-eae5cbc75de5 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:44:54.257822 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-b2abbcb5-74ef-420f-97e2-eae5cbc75de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:54.353218 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 965/1930] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:44:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:44:54.361303 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee3387-605a-4a45-83c1-1f020fc9fb9f None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:54.362240 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee3387-605a-4a45-83c1-1f020fc9fb9f None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:54.362651 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee3387-605a-4a45-83c1-1f020fc9fb9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:54.383153 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-9fee3387-605a-4a45-83c1-1f020fc9fb9f None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:44:54.384505 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-9fee3387-605a-4a45-83c1-1f020fc9fb9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:54.393315 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 966/1931] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:44:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:44:54.397654 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2abbcb5-74ef-420f-97e2-eae5cbc75de5 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:54.408275 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2abbcb5-74ef-420f-97e2-eae5cbc75de5 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:54.420204 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a5154-a8ea-40ec-b385-df27c13cba9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:54.420846 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a5154-a8ea-40ec-b385-df27c13cba9a admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:54.421152 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a5154-a8ea-40ec-b385-df27c13cba9a admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:54.421423 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a5154-a8ea-40ec-b385-df27c13cba9a admin admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:54.422075 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a5154-a8ea-40ec-b385-df27c13cba9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:44:54.422377 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a5154-a8ea-40ec-b385-df27c13cba9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:54.424035 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a5154-a8ea-40ec-b385-df27c13cba9a admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:54.439441 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 966/1932] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:44:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:44:54.441273 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee3387-605a-4a45-83c1-1f020fc9fb9f None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:54.451895 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee3387-605a-4a45-83c1-1f020fc9fb9f None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:54.456762 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e09254-9fd9-49c8-abf2-554796c17786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:54.457372 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e09254-9fd9-49c8-abf2-554796c17786 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:54.457655 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e09254-9fd9-49c8-abf2-554796c17786 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:54.457929 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e09254-9fd9-49c8-abf2-554796c17786 admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:54.458558 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e09254-9fd9-49c8-abf2-554796c17786 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:54.460498 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e09254-9fd9-49c8-abf2-554796c17786 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:54.493781 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 967/1933] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:44:54 2022] POST /identity/v3/projects => generated 401 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:44:54.498616 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a5154-a8ea-40ec-b385-df27c13cba9a admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:54.510911 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a5154-a8ea-40ec-b385-df27c13cba9a admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:54.539654 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:44:54.574451 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3479705d-4494-4137-bc62-934a23b26a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:54.575319 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479705d-4494-4137-bc62-934a23b26a99 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:54.575600 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479705d-4494-4137-bc62-934a23b26a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:54.575869 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479705d-4494-4137-bc62-934a23b26a99 admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:54.576542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3479705d-4494-4137-bc62-934a23b26a99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:54.578460 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3479705d-4494-4137-bc62-934a23b26a99 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:54.601316 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 967/1934] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:44:54 2022] POST /identity/v3/users => generated 301 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:44:54.605833 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e09254-9fd9-49c8-abf2-554796c17786 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:54.616544 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e09254-9fd9-49c8-abf2-554796c17786 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:54.621159 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222f60a-d527-4aa3-83c6-7682ee709e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:54.621756 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222f60a-d527-4aa3-83c6-7682ee709e2c admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:54.622036 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222f60a-d527-4aa3-83c6-7682ee709e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:54.622310 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222f60a-d527-4aa3-83c6-7682ee709e2c admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:54.622956 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4222f60a-d527-4aa3-83c6-7682ee709e2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:54.624632 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4222f60a-d527-4aa3-83c6-7682ee709e2c admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:54.635159 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 968/1935] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:44:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:44:54.640073 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3479705d-4494-4137-bc62-934a23b26a99 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:54.649991 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3479705d-4494-4137-bc62-934a23b26a99 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:54.655123 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62104101-a9a1-423a-9d95-787fa687f1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:54.655713 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62104101-a9a1-423a-9d95-787fa687f1bd admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:54.656011 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62104101-a9a1-423a-9d95-787fa687f1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:54.656286 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62104101-a9a1-423a-9d95-787fa687f1bd admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:54.657189 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62104101-a9a1-423a-9d95-787fa687f1bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:54.658777 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62104101-a9a1-423a-9d95-787fa687f1bd admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:54.668982 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 968/1936] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:44:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:44:54.672907 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222f60a-d527-4aa3-83c6-7682ee709e2c admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:54.683068 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222f60a-d527-4aa3-83c6-7682ee709e2c admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:54.687596 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b634a273-69a2-46d2-9edd-78166f6e1e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:54.688268 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634a273-69a2-46d2-9edd-78166f6e1e83 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:54.688556 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634a273-69a2-46d2-9edd-78166f6e1e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:54.688821 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634a273-69a2-46d2-9edd-78166f6e1e83 admin admin] PATH_INFO: `/v3/projects/41b712b7f7a44dad8f2ab48460260f13/users/7d73f4f2c8934c3a927027a0893055c7/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:54.704164 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b634a273-69a2-46d2-9edd-78166f6e1e83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41b712b7f7a44dad8f2ab48460260f13, user_id=7d73f4f2c8934c3a927027a0893055c7, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:54.706865 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b634a273-69a2-46d2-9edd-78166f6e1e83 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:54.721104 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 969/1937] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:44:54 2022] PUT /identity/v3/projects/41b712b7f7a44dad8f2ab48460260f13/users/7d73f4f2c8934c3a927027a0893055c7/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:44:54.732440 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ef42a-7be4-4d23-b626-fcb87f2fa0c3 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:54.732724 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ef42a-7be4-4d23-b626-fcb87f2fa0c3 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:54.732995 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ef42a-7be4-4d23-b626-fcb87f2fa0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:54.760368 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-021ef42a-7be4-4d23-b626-fcb87f2fa0c3 None None] MFA Rules not processed for user `7d73f4f2c8934c3a927027a0893055c7`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:44:54.762102 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-021ef42a-7be4-4d23-b626-fcb87f2fa0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:54.830213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 969/1938] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:44:54 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:44:54.835094 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b634a273-69a2-46d2-9edd-78166f6e1e83 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:54.898416 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b634a273-69a2-46d2-9edd-78166f6e1e83 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:54.903090 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c363c-c01e-458e-ad38-b557afdad1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:54.903703 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c363c-c01e-458e-ad38-b557afdad1ab admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:54.904002 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c363c-c01e-458e-ad38-b557afdad1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:54.904281 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c363c-c01e-458e-ad38-b557afdad1ab admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:54.904924 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0c363c-c01e-458e-ad38-b557afdad1ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:54.906846 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0c363c-c01e-458e-ad38-b557afdad1ab admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:54.935216 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 970/1939] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:44:54 2022] POST /identity/v3/projects => generated 401 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:44:54.937270 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beceab44-390c-4e11-963f-458884fd01d7 None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:54.937628 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beceab44-390c-4e11-963f-458884fd01d7 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:54.938025 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beceab44-390c-4e11-963f-458884fd01d7 None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:54.939132 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c363c-c01e-458e-ad38-b557afdad1ab admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:54.940295 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 970/1940] 10.222.0.26 () {54 vars in 921 bytes} [Wed Aug 17 23:44:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:44:54.973695 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b45e16-477f-4508-9233-5e173b4c26a1 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:54.974318 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b45e16-477f-4508-9233-5e173b4c26a1 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:54.975134 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b45e16-477f-4508-9233-5e173b4c26a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:54.986813 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:44:55.015320 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-27b45e16-477f-4508-9233-5e173b4c26a1 None None] MFA Rules not processed for user `1fc57d911a56423e96c736bc96270ba8`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:44:55.016555 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-27b45e16-477f-4508-9233-5e173b4c26a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:55.017743 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c363c-c01e-458e-ad38-b557afdad1ab admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:55.025086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc42fc1-4c62-4115-b355-fd8257d582fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:55.026136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc42fc1-4c62-4115-b355-fd8257d582fd admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:55.027346 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc42fc1-4c62-4115-b355-fd8257d582fd admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:55.027824 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc42fc1-4c62-4115-b355-fd8257d582fd admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:55.028723 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc42fc1-4c62-4115-b355-fd8257d582fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:55.030844 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc42fc1-4c62-4115-b355-fd8257d582fd admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:55.083934 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 971/1941] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:44:54 2022] POST /identity/v3/users => generated 307 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:44:55.085368 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc42fc1-4c62-4115-b355-fd8257d582fd admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:55.101947 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 971/1942] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Aug 17 23:44:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:44:55.104618 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc42fc1-4c62-4115-b355-fd8257d582fd admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:55.112200 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b08963-cd45-4031-b00a-ecf21a63ee04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:55.113283 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b08963-cd45-4031-b00a-ecf21a63ee04 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:55.113283 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b08963-cd45-4031-b00a-ecf21a63ee04 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:55.114049 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b08963-cd45-4031-b00a-ecf21a63ee04 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:55.114801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b08963-cd45-4031-b00a-ecf21a63ee04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:55.116475 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b08963-cd45-4031-b00a-ecf21a63ee04 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:55.138549 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 972/1943] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:44:55.139769 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b45e16-477f-4508-9233-5e173b4c26a1 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:55.140792 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b08963-cd45-4031-b00a-ecf21a63ee04 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:55.145405 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-27b45e16-477f-4508-9233-5e173b4c26a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:55.151531 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b08963-cd45-4031-b00a-ecf21a63ee04 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:55.158734 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70cf1fe-3532-48c0-869b-b4da302047b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:55.159315 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70cf1fe-3532-48c0-869b-b4da302047b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:55.159653 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70cf1fe-3532-48c0-869b-b4da302047b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:55.159980 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70cf1fe-3532-48c0-869b-b4da302047b9 admin admin] PATH_INFO: `/v3/projects/ea06144937534994908fb8349b454918/users/e8ecaa094aba4624b4408df49341c217/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:55.182461 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70cf1fe-3532-48c0-869b-b4da302047b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea06144937534994908fb8349b454918, user_id=e8ecaa094aba4624b4408df49341c217, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:55.186140 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70cf1fe-3532-48c0-869b-b4da302047b9 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:55.186559 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b45e16-477f-4508-9233-5e173b4c26a1 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:55.194990 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790357d8-8a03-430e-b045-b396c85c65b7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc57d911a56423e96c736bc96270ba8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:55.196241 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790357d8-8a03-430e-b045-b396c85c65b7 service placement] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:55.196868 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790357d8-8a03-430e-b045-b396c85c65b7 service placement] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:55.197379 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790357d8-8a03-430e-b045-b396c85c65b7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:55.208096 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 973/1944] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:44:55 2022] PUT /identity/v3/projects/ea06144937534994908fb8349b454918/users/e8ecaa094aba4624b4408df49341c217/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:44:55.209830 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70cf1fe-3532-48c0-869b-b4da302047b9 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:55.211908 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790357d8-8a03-430e-b045-b396c85c65b7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:55.213740 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790357d8-8a03-430e-b045-b396c85c65b7 service placement] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:55.225147 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 972/1945] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:44:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:44:55.303781 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70cf1fe-3532-48c0-869b-b4da302047b9 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:55.313898 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62ae13-6de3-4874-95c5-62494a46e908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:55.314631 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62ae13-6de3-4874-95c5-62494a46e908 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:55.315025 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62ae13-6de3-4874-95c5-62494a46e908 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:55.315495 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62ae13-6de3-4874-95c5-62494a46e908 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:55.316640 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a62ae13-6de3-4874-95c5-62494a46e908 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:55.319826 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a62ae13-6de3-4874-95c5-62494a46e908 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:55.333202 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 974/1946] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:44:55.336882 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790357d8-8a03-430e-b045-b396c85c65b7 service placement] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:55.362566 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790357d8-8a03-430e-b045-b396c85c65b7 service placement] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:55.371533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57143d47-36d6-4d3c-9536-4dbeba410b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:55.372236 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57143d47-36d6-4d3c-9536-4dbeba410b7f admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:55.372526 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57143d47-36d6-4d3c-9536-4dbeba410b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:55.372790 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57143d47-36d6-4d3c-9536-4dbeba410b7f admin admin] PATH_INFO: `/v3/projects/ea06144937534994908fb8349b454918/users/e8ecaa094aba4624b4408df49341c217/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:55.374796 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57143d47-36d6-4d3c-9536-4dbeba410b7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea06144937534994908fb8349b454918, user_id=e8ecaa094aba4624b4408df49341c217, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:55.377741 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57143d47-36d6-4d3c-9536-4dbeba410b7f admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:55.390835 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 973/1947] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:44:55 2022] PUT /identity/v3/projects/ea06144937534994908fb8349b454918/users/e8ecaa094aba4624b4408df49341c217/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:44:55.401380 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd176f12-edf5-4646-b32b-54f7f8d15148 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:55.401895 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd176f12-edf5-4646-b32b-54f7f8d15148 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:55.402143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd176f12-edf5-4646-b32b-54f7f8d15148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:55.447614 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-fd176f12-edf5-4646-b32b-54f7f8d15148 None None] MFA Rules not processed for user `e8ecaa094aba4624b4408df49341c217`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:44:55.450052 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-fd176f12-edf5-4646-b32b-54f7f8d15148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:55.568697 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 975/1948] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:44:55 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:44:55.650597 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57143d47-36d6-4d3c-9536-4dbeba410b7f admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:55.653504 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-57143d47-36d6-4d3c-9536-4dbeba410b7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:55.666320 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:44:55.729057 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57143d47-36d6-4d3c-9536-4dbeba410b7f admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:55.737456 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a165e1-a0df-490a-98db-449f5d09e7dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:55.738604 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a165e1-a0df-490a-98db-449f5d09e7dd service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:55.739062 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a165e1-a0df-490a-98db-449f5d09e7dd service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:55.739337 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a165e1-a0df-490a-98db-449f5d09e7dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:55.751262 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a165e1-a0df-490a-98db-449f5d09e7dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:55.753046 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a165e1-a0df-490a-98db-449f5d09e7dd service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:55.763195 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 974/1949] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:44:55 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:44:55.824484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd176f12-edf5-4646-b32b-54f7f8d15148 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:55.826755 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-fd176f12-edf5-4646-b32b-54f7f8d15148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:55.844778 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a165e1-a0df-490a-98db-449f5d09e7dd service cinder] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:55.848304 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:44:55.858947 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a165e1-a0df-490a-98db-449f5d09e7dd service cinder] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:55.862625 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a4c64-2d10-4a16-bcfa-cb2dd800648b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:55.863299 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a4c64-2d10-4a16-bcfa-cb2dd800648b service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:55.863651 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a4c64-2d10-4a16-bcfa-cb2dd800648b service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:55.864021 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a4c64-2d10-4a16-bcfa-cb2dd800648b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:55.907673 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd176f12-edf5-4646-b32b-54f7f8d15148 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:55.913418 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8317d-a8bd-4c1d-a5c1-c22844a6d198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:55.914169 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8317d-a8bd-4c1d-a5c1-c22844a6d198 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:55.914457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8317d-a8bd-4c1d-a5c1-c22844a6d198 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:55.914738 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8317d-a8bd-4c1d-a5c1-c22844a6d198 admin admin] PATH_INFO: `/v3/users/ee47903c6ad741bcaa371ae414a793b9` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:55.915901 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f8317d-a8bd-4c1d-a5c1-c22844a6d198 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee47903c6ad741bcaa371ae414a793b9)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:55.918650 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f8317d-a8bd-4c1d-a5c1-c22844a6d198 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:55.926772 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a4c64-2d10-4a16-bcfa-cb2dd800648b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:55.928622 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a4c64-2d10-4a16-bcfa-cb2dd800648b service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:55.939044 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 975/1950] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:44:55 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 23:44:55.990016 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-20f8317d-a8bd-4c1d-a5c1-c22844a6d198 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee47903c6ad741bcaa371ae414a793b9'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:55.990948 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-20f8317d-a8bd-4c1d-a5c1-c22844a6d198 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee47903c6ad741bcaa371ae414a793b9'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:55.991611 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-20f8317d-a8bd-4c1d-a5c1-c22844a6d198 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee47903c6ad741bcaa371ae414a793b9'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:56.003633 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-20f8317d-a8bd-4c1d-a5c1-c22844a6d198 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee47903c6ad741bcaa371ae414a793b9'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:56.021133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-20f8317d-a8bd-4c1d-a5c1-c22844a6d198 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee47903c6ad741bcaa371ae414a793b9'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:56.044132 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 976/1951] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:44:55 2022] DELETE /identity/v3/users/ee47903c6ad741bcaa371ae414a793b9 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:44:56.438686 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a4c64-2d10-4a16-bcfa-cb2dd800648b service cinder] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:56.475880 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a4c64-2d10-4a16-bcfa-cb2dd800648b service cinder] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:56.547148 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d0cf02-5936-436b-bd8f-f0e7e75fa209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:56.547148 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0cf02-5936-436b-bd8f-f0e7e75fa209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:56.547365 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0cf02-5936-436b-bd8f-f0e7e75fa209 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:56.547635 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0cf02-5936-436b-bd8f-f0e7e75fa209 admin admin] PATH_INFO: `/v3/projects/135edfbc128148279cdbf0eb10ad05a2` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:56.549088 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d0cf02-5936-436b-bd8f-f0e7e75fa209 admin admin] RBAC: Authorizing `identity:delete_project(project_id=135edfbc128148279cdbf0eb10ad05a2)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:56.552050 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d0cf02-5936-436b-bd8f-f0e7e75fa209 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:56.602105 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-96d0cf02-5936-436b-bd8f-f0e7e75fa209 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '135edfbc128148279cdbf0eb10ad05a2', 'request_id': 'req-96d0cf02-5936-436b-bd8f-f0e7e75fa209'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:56.603512 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:44:56.608963 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-96d0cf02-5936-436b-bd8f-f0e7e75fa209 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '135edfbc128148279cdbf0eb10ad05a2', 'request_id': 'req-96d0cf02-5936-436b-bd8f-f0e7e75fa209'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:56.618990 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-96d0cf02-5936-436b-bd8f-f0e7e75fa209 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '135edfbc128148279cdbf0eb10ad05a2', 'request_id': 'req-96d0cf02-5936-436b-bd8f-f0e7e75fa209'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:56.632174 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-96d0cf02-5936-436b-bd8f-f0e7e75fa209 admin admin] The token cache is being invalidate because project 135edfbc128148279cdbf0eb10ad05a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:44:56.632465 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-96d0cf02-5936-436b-bd8f-f0e7e75fa209 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:56.633131 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-96d0cf02-5936-436b-bd8f-f0e7e75fa209 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:56.634867 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 976/1952] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:44:56 2022] DELETE /identity/v3/projects/135edfbc128148279cdbf0eb10ad05a2 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:44:56.639806 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8317d-a8bd-4c1d-a5c1-c22844a6d198 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:56.642711 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-20f8317d-a8bd-4c1d-a5c1-c22844a6d198 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:56.660041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8317d-a8bd-4c1d-a5c1-c22844a6d198 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:56.725744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704eff6-b869-4fb0-95a0-7fdcf5ffba23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:56.726309 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704eff6-b869-4fb0-95a0-7fdcf5ffba23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:56.726588 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704eff6-b869-4fb0-95a0-7fdcf5ffba23 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:56.726860 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704eff6-b869-4fb0-95a0-7fdcf5ffba23 admin admin] PATH_INFO: `/v3/users/ac26cb13b643491cb7206ac74a7a5914` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:56.728201 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5704eff6-b869-4fb0-95a0-7fdcf5ffba23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac26cb13b643491cb7206ac74a7a5914)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:56.730845 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5704eff6-b869-4fb0-95a0-7fdcf5ffba23 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:56.776456 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:44:56.795014 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-5704eff6-b869-4fb0-95a0-7fdcf5ffba23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac26cb13b643491cb7206ac74a7a5914'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:56.796565 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-5704eff6-b869-4fb0-95a0-7fdcf5ffba23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac26cb13b643491cb7206ac74a7a5914'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:56.797440 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-5704eff6-b869-4fb0-95a0-7fdcf5ffba23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac26cb13b643491cb7206ac74a7a5914'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:56.806969 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-5704eff6-b869-4fb0-95a0-7fdcf5ffba23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac26cb13b643491cb7206ac74a7a5914'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:56.815158 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-5704eff6-b869-4fb0-95a0-7fdcf5ffba23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac26cb13b643491cb7206ac74a7a5914'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:56.832602 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 977/1953] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:44:56 2022] DELETE /identity/v3/users/ac26cb13b643491cb7206ac74a7a5914 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:44:57.018295 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d0cf02-5936-436b-bd8f-f0e7e75fa209 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:57.019893 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-96d0cf02-5936-436b-bd8f-f0e7e75fa209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:57.043083 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d0cf02-5936-436b-bd8f-f0e7e75fa209 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:57.105797 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c509137-10b8-4383-9cb7-4ad797b3c4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:57.106458 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c509137-10b8-4383-9cb7-4ad797b3c4f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:57.106734 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c509137-10b8-4383-9cb7-4ad797b3c4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:57.107032 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c509137-10b8-4383-9cb7-4ad797b3c4f5 admin admin] PATH_INFO: `/v3/projects/7f2944f3ccf74c45a9ca6105199ab1cd` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:57.108150 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c509137-10b8-4383-9cb7-4ad797b3c4f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f2944f3ccf74c45a9ca6105199ab1cd)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:57.110298 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c509137-10b8-4383-9cb7-4ad797b3c4f5 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:57.166015 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-7c509137-10b8-4383-9cb7-4ad797b3c4f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f2944f3ccf74c45a9ca6105199ab1cd', 'request_id': 'req-7c509137-10b8-4383-9cb7-4ad797b3c4f5'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:57.173021 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-7c509137-10b8-4383-9cb7-4ad797b3c4f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f2944f3ccf74c45a9ca6105199ab1cd', 'request_id': 'req-7c509137-10b8-4383-9cb7-4ad797b3c4f5'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:57.182997 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-7c509137-10b8-4383-9cb7-4ad797b3c4f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f2944f3ccf74c45a9ca6105199ab1cd', 'request_id': 'req-7c509137-10b8-4383-9cb7-4ad797b3c4f5'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:57.195080 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-7c509137-10b8-4383-9cb7-4ad797b3c4f5 admin admin] The token cache is being invalidate because project 7f2944f3ccf74c45a9ca6105199ab1cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:44:57.195473 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-7c509137-10b8-4383-9cb7-4ad797b3c4f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:57.196366 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-7c509137-10b8-4383-9cb7-4ad797b3c4f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:57.201326 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 977/1954] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:44:57 2022] DELETE /identity/v3/projects/7f2944f3ccf74c45a9ca6105199ab1cd => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:44:57.203031 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a40f94-cf34-4767-9eed-fb4668d96a23 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:57.203379 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a40f94-cf34-4767-9eed-fb4668d96a23 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:57.203728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a40f94-cf34-4767-9eed-fb4668d96a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:57.223824 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-e2a40f94-cf34-4767-9eed-fb4668d96a23 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:44:57.225164 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-e2a40f94-cf34-4767-9eed-fb4668d96a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:57.284023 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 978/1955] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:44:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:44:57.292098 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e92037-b804-4060-9307-42cd2055fadd None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:57.292407 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e92037-b804-4060-9307-42cd2055fadd None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:57.293267 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e92037-b804-4060-9307-42cd2055fadd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:57.312585 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-62e92037-b804-4060-9307-42cd2055fadd None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:44:57.313834 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-62e92037-b804-4060-9307-42cd2055fadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:57.321473 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 978/1956] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:44:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:44:57.325653 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a40f94-cf34-4767-9eed-fb4668d96a23 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:57.335236 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a40f94-cf34-4767-9eed-fb4668d96a23 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:57.340205 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450bc6a-06c6-4e35-a342-982821cd4c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:57.340791 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450bc6a-06c6-4e35-a342-982821cd4c8c admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:57.341072 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450bc6a-06c6-4e35-a342-982821cd4c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:57.341340 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450bc6a-06c6-4e35-a342-982821cd4c8c admin admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:57.341977 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3450bc6a-06c6-4e35-a342-982821cd4c8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:44:57.342274 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3450bc6a-06c6-4e35-a342-982821cd4c8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:57.343893 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3450bc6a-06c6-4e35-a342-982821cd4c8c admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:57.355993 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 979/1957] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:44:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:44:57.360072 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e92037-b804-4060-9307-42cd2055fadd None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:57.370059 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e92037-b804-4060-9307-42cd2055fadd None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:57.375987 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75067e1a-9df0-443d-a434-0df0f4498a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:57.376867 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75067e1a-9df0-443d-a434-0df0f4498a30 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:57.377236 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75067e1a-9df0-443d-a434-0df0f4498a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:57.377603 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75067e1a-9df0-443d-a434-0df0f4498a30 admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:57.378581 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75067e1a-9df0-443d-a434-0df0f4498a30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:57.382338 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75067e1a-9df0-443d-a434-0df0f4498a30 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:57.423762 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 979/1958] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:44:57 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:44:57.425465 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450bc6a-06c6-4e35-a342-982821cd4c8c admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:57.437566 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450bc6a-06c6-4e35-a342-982821cd4c8c admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:57.515778 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec99ea-48a1-41e5-91ee-a9f600b53349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:57.516424 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec99ea-48a1-41e5-91ee-a9f600b53349 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:57.516709 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec99ea-48a1-41e5-91ee-a9f600b53349 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:57.516983 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec99ea-48a1-41e5-91ee-a9f600b53349 admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:57.517610 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec99ea-48a1-41e5-91ee-a9f600b53349 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:57.519520 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec99ea-48a1-41e5-91ee-a9f600b53349 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:57.542380 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 980/1959] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:44:57 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:44:57.546492 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75067e1a-9df0-443d-a434-0df0f4498a30 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:57.557628 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75067e1a-9df0-443d-a434-0df0f4498a30 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:57.562353 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb6beb-78e2-4e6a-a7fd-c4875998a931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:57.563109 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb6beb-78e2-4e6a-a7fd-c4875998a931 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:57.563460 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb6beb-78e2-4e6a-a7fd-c4875998a931 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:57.563806 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb6beb-78e2-4e6a-a7fd-c4875998a931 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:57.564582 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eb6beb-78e2-4e6a-a7fd-c4875998a931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:57.566325 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eb6beb-78e2-4e6a-a7fd-c4875998a931 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:57.579486 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 980/1960] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:44:57.583579 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec99ea-48a1-41e5-91ee-a9f600b53349 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:57.594319 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec99ea-48a1-41e5-91ee-a9f600b53349 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:57.598816 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6801e9-e80b-4c91-b171-27bdffa6d23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:57.599454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6801e9-e80b-4c91-b171-27bdffa6d23b admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:57.599736 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6801e9-e80b-4c91-b171-27bdffa6d23b admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:57.600046 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6801e9-e80b-4c91-b171-27bdffa6d23b admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:57.600720 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6801e9-e80b-4c91-b171-27bdffa6d23b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:57.602289 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6801e9-e80b-4c91-b171-27bdffa6d23b admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:57.613079 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 981/1961] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:44:57.617131 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb6beb-78e2-4e6a-a7fd-c4875998a931 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:57.627391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb6beb-78e2-4e6a-a7fd-c4875998a931 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:57.632148 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab1e2e-6e79-4545-98f4-ed0ce6d539aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:57.632834 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab1e2e-6e79-4545-98f4-ed0ce6d539aa admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:57.633105 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab1e2e-6e79-4545-98f4-ed0ce6d539aa admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:57.633377 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab1e2e-6e79-4545-98f4-ed0ce6d539aa admin admin] PATH_INFO: `/v3/projects/7eacd61345c1462ba5eb1b220602481e/users/69c1025cd09d456fa518863f84498125/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:57.649073 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab1e2e-6e79-4545-98f4-ed0ce6d539aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=7eacd61345c1462ba5eb1b220602481e, user_id=69c1025cd09d456fa518863f84498125, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:57.651743 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab1e2e-6e79-4545-98f4-ed0ce6d539aa admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:57.665405 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 981/1962] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:44:57 2022] PUT /identity/v3/projects/7eacd61345c1462ba5eb1b220602481e/users/69c1025cd09d456fa518863f84498125/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:44:57.675706 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd0723-a468-48ca-b4f3-7045eab242ae None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:57.676186 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd0723-a468-48ca-b4f3-7045eab242ae None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:57.676291 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd0723-a468-48ca-b4f3-7045eab242ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:57.704014 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-4efd0723-a468-48ca-b4f3-7045eab242ae None None] MFA Rules not processed for user `69c1025cd09d456fa518863f84498125`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:44:57.705228 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-4efd0723-a468-48ca-b4f3-7045eab242ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:57.709099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:44:57.771293 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 982/1963] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:44:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:44:57.784338 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab1e2e-6e79-4545-98f4-ed0ce6d539aa admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:57.786388 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-ecab1e2e-6e79-4545-98f4-ed0ce6d539aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:57.793184 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:44:57.847857 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab1e2e-6e79-4545-98f4-ed0ce6d539aa admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:57.851536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efdef07-f539-4632-a486-9f07af2dab32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:57.852165 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efdef07-f539-4632-a486-9f07af2dab32 service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:57.852447 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efdef07-f539-4632-a486-9f07af2dab32 service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:57.852718 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efdef07-f539-4632-a486-9f07af2dab32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:57.861521 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efdef07-f539-4632-a486-9f07af2dab32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:57.862966 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efdef07-f539-4632-a486-9f07af2dab32 service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:57.872486 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 982/1964] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:44:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:44:57.892075 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd0723-a468-48ca-b4f3-7045eab242ae None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:57.954589 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd0723-a468-48ca-b4f3-7045eab242ae None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:57.959322 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9ed7b-17f2-4767-94bc-04ddc7b9edca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:57.959982 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9ed7b-17f2-4767-94bc-04ddc7b9edca admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:57.960286 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9ed7b-17f2-4767-94bc-04ddc7b9edca admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:57.960561 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9ed7b-17f2-4767-94bc-04ddc7b9edca admin admin] PATH_INFO: `/v3/users/69c1025cd09d456fa518863f84498125` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:57.961723 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e9ed7b-17f2-4767-94bc-04ddc7b9edca admin admin] RBAC: Authorizing `identity:delete_user(user_id=69c1025cd09d456fa518863f84498125)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:57.963620 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e9ed7b-17f2-4767-94bc-04ddc7b9edca admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:58.008745 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-a6e9ed7b-17f2-4767-94bc-04ddc7b9edca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69c1025cd09d456fa518863f84498125'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:58.009407 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-a6e9ed7b-17f2-4767-94bc-04ddc7b9edca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69c1025cd09d456fa518863f84498125'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:58.010024 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-a6e9ed7b-17f2-4767-94bc-04ddc7b9edca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69c1025cd09d456fa518863f84498125'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:58.017709 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-a6e9ed7b-17f2-4767-94bc-04ddc7b9edca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69c1025cd09d456fa518863f84498125'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:58.024378 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-a6e9ed7b-17f2-4767-94bc-04ddc7b9edca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69c1025cd09d456fa518863f84498125'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:58.041087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 983/1965] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:44:57 2022] DELETE /identity/v3/users/69c1025cd09d456fa518863f84498125 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:44:58.045678 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f41980-d5c0-4d43-a90a-a2b000ab77b0 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:58.045903 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f41980-d5c0-4d43-a90a-a2b000ab77b0 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:58.046175 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f41980-d5c0-4d43-a90a-a2b000ab77b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:58.065130 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-d0f41980-d5c0-4d43-a90a-a2b000ab77b0 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:44:58.066458 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-d0f41980-d5c0-4d43-a90a-a2b000ab77b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:58.125229 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 983/1966] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:44:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:44:58.134924 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9ed7b-17f2-4767-94bc-04ddc7b9edca admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:58.136977 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-a6e9ed7b-17f2-4767-94bc-04ddc7b9edca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:58.164535 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:44:58.194723 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9ed7b-17f2-4767-94bc-04ddc7b9edca admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:58.198343 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7a447-8e5c-4669-955c-ae237ebf135d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:58.198953 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7a447-8e5c-4669-955c-ae237ebf135d service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:58.199222 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7a447-8e5c-4669-955c-ae237ebf135d service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:58.199495 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7a447-8e5c-4669-955c-ae237ebf135d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:58.208540 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b7a447-8e5c-4669-955c-ae237ebf135d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:58.210189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b7a447-8e5c-4669-955c-ae237ebf135d service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:58.219947 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 984/1967] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:44:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:44:58.484144 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f41980-d5c0-4d43-a90a-a2b000ab77b0 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:58.485721 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-d0f41980-d5c0-4d43-a90a-a2b000ab77b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:58.500419 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f41980-d5c0-4d43-a90a-a2b000ab77b0 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:58.505094 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07feabc9-afec-4a6d-ad3f-9018d7936d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:58.505845 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07feabc9-afec-4a6d-ad3f-9018d7936d5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:58.506123 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07feabc9-afec-4a6d-ad3f-9018d7936d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:58.506389 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07feabc9-afec-4a6d-ad3f-9018d7936d5a admin admin] PATH_INFO: `/v3/projects/7eacd61345c1462ba5eb1b220602481e` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:58.507518 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07feabc9-afec-4a6d-ad3f-9018d7936d5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7eacd61345c1462ba5eb1b220602481e)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:58.509465 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07feabc9-afec-4a6d-ad3f-9018d7936d5a admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:58.557168 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-07feabc9-afec-4a6d-ad3f-9018d7936d5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7eacd61345c1462ba5eb1b220602481e', 'request_id': 'req-07feabc9-afec-4a6d-ad3f-9018d7936d5a'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:58.563333 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-07feabc9-afec-4a6d-ad3f-9018d7936d5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7eacd61345c1462ba5eb1b220602481e', 'request_id': 'req-07feabc9-afec-4a6d-ad3f-9018d7936d5a'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:58.570037 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-07feabc9-afec-4a6d-ad3f-9018d7936d5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7eacd61345c1462ba5eb1b220602481e', 'request_id': 'req-07feabc9-afec-4a6d-ad3f-9018d7936d5a'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:58.581146 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-07feabc9-afec-4a6d-ad3f-9018d7936d5a admin admin] The token cache is being invalidate because project 7eacd61345c1462ba5eb1b220602481e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:44:58.581461 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-07feabc9-afec-4a6d-ad3f-9018d7936d5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:58.582094 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-07feabc9-afec-4a6d-ad3f-9018d7936d5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:44:58.583942 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 984/1968] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:44:58 2022] DELETE /identity/v3/projects/7eacd61345c1462ba5eb1b220602481e => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:44:58.588935 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0cfb07-a161-4090-a005-623aef28807d None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:58.589252 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0cfb07-a161-4090-a005-623aef28807d None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:58.589642 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0cfb07-a161-4090-a005-623aef28807d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:58.611776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-ed0cfb07-a161-4090-a005-623aef28807d None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:44:58.613033 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-ed0cfb07-a161-4090-a005-623aef28807d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:58.680772 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 985/1969] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:44:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:44:58.689677 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab251c-0cdf-4b8e-b3ce-fce5af01b809 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:58.689965 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab251c-0cdf-4b8e-b3ce-fce5af01b809 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:58.690244 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab251c-0cdf-4b8e-b3ce-fce5af01b809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:58.710031 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-a7ab251c-0cdf-4b8e-b3ce-fce5af01b809 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:44:58.711250 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-a7ab251c-0cdf-4b8e-b3ce-fce5af01b809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:58.723187 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 985/1970] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:44:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:44:58.723860 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0cfb07-a161-4090-a005-623aef28807d None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:58.734845 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0cfb07-a161-4090-a005-623aef28807d None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:58.739779 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dc79fb-3871-4390-b521-be5cb247a697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:58.740378 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc79fb-3871-4390-b521-be5cb247a697 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:58.740721 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc79fb-3871-4390-b521-be5cb247a697 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:58.741055 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc79fb-3871-4390-b521-be5cb247a697 admin admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:58.741978 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dc79fb-3871-4390-b521-be5cb247a697 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:44:58.742498 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dc79fb-3871-4390-b521-be5cb247a697 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:58.744369 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dc79fb-3871-4390-b521-be5cb247a697 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:58.765017 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 986/1971] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:44:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:44:58.766903 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab251c-0cdf-4b8e-b3ce-fce5af01b809 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:58.776717 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab251c-0cdf-4b8e-b3ce-fce5af01b809 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:58.781799 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039082d-9b52-4f77-9144-ed50100badd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:58.782498 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039082d-9b52-4f77-9144-ed50100badd2 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:58.782687 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039082d-9b52-4f77-9144-ed50100badd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:58.782958 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039082d-9b52-4f77-9144-ed50100badd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:58.783592 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9039082d-9b52-4f77-9144-ed50100badd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:58.785875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9039082d-9b52-4f77-9144-ed50100badd2 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:58.822169 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 986/1972] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:44:58 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:44:58.826845 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dc79fb-3871-4390-b521-be5cb247a697 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:58.836772 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dc79fb-3871-4390-b521-be5cb247a697 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:58.894580 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917a057-1d26-43f2-b313-77ebabb3c3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:58.895220 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917a057-1d26-43f2-b313-77ebabb3c3af admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:58.895502 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917a057-1d26-43f2-b313-77ebabb3c3af admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:58.895775 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917a057-1d26-43f2-b313-77ebabb3c3af admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:58.896444 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c917a057-1d26-43f2-b313-77ebabb3c3af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:58.898355 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c917a057-1d26-43f2-b313-77ebabb3c3af admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:58.919732 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 987/1973] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:44:58 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:44:58.923668 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039082d-9b52-4f77-9144-ed50100badd2 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:58.929098 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:44:58.938514 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039082d-9b52-4f77-9144-ed50100badd2 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:58.944400 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0489a1-e6f5-467c-852e-8282481c6028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:58.945004 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0489a1-e6f5-467c-852e-8282481c6028 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:58.945282 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0489a1-e6f5-467c-852e-8282481c6028 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:58.945549 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0489a1-e6f5-467c-852e-8282481c6028 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:58.946182 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0489a1-e6f5-467c-852e-8282481c6028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:58.947778 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0489a1-e6f5-467c-852e-8282481c6028 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:58.958349 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 987/1974] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:44:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 23:44:58.962439 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917a057-1d26-43f2-b313-77ebabb3c3af admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:58.973771 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917a057-1d26-43f2-b313-77ebabb3c3af admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:58.978465 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc401c-6286-481c-80ba-73eb8bb9befb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:58.979148 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc401c-6286-481c-80ba-73eb8bb9befb admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:58.979352 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc401c-6286-481c-80ba-73eb8bb9befb admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:58.979629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc401c-6286-481c-80ba-73eb8bb9befb admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:58.980870 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdc401c-6286-481c-80ba-73eb8bb9befb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:58.982504 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdc401c-6286-481c-80ba-73eb8bb9befb admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:58.999775 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 988/1975] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:44:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:44:59.001106 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0489a1-e6f5-467c-852e-8282481c6028 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:59.017060 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0489a1-e6f5-467c-852e-8282481c6028 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:59.022325 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ec51a-10df-43c5-a909-1de11ed54f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:59.023005 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ec51a-10df-43c5-a909-1de11ed54f52 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:59.023289 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ec51a-10df-43c5-a909-1de11ed54f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:59.023565 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ec51a-10df-43c5-a909-1de11ed54f52 admin admin] PATH_INFO: `/v3/projects/b391ac03862c41279b782a53bf0c7310/users/d9db8c44b6c64ff7b54159278400063c/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:59.040905 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ec51a-10df-43c5-a909-1de11ed54f52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b391ac03862c41279b782a53bf0c7310, user_id=d9db8c44b6c64ff7b54159278400063c, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:59.044342 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ec51a-10df-43c5-a909-1de11ed54f52 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:59.059317 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 988/1976] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:44:59 2022] PUT /identity/v3/projects/b391ac03862c41279b782a53bf0c7310/users/d9db8c44b6c64ff7b54159278400063c/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:44:59.069608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e73ad9-bc11-4b23-b523-5d6f4b365ec8 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:59.069990 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e73ad9-bc11-4b23-b523-5d6f4b365ec8 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:59.070540 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e73ad9-bc11-4b23-b523-5d6f4b365ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:59.101091 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-23e73ad9-bc11-4b23-b523-5d6f4b365ec8 None None] MFA Rules not processed for user `d9db8c44b6c64ff7b54159278400063c`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:44:59.102425 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-23e73ad9-bc11-4b23-b523-5d6f4b365ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:59.124972 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:44:59.176747 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 989/1977] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:44:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:44:59.187355 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ec51a-10df-43c5-a909-1de11ed54f52 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:44:59.189003 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-113ec51a-10df-43c5-a909-1de11ed54f52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:44:59.244646 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ec51a-10df-43c5-a909-1de11ed54f52 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:44:59.248203 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a53e7d-219e-422f-9ae7-6cf32282fd18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:44:59.248806 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a53e7d-219e-422f-9ae7-6cf32282fd18 service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:44:59.249081 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a53e7d-219e-422f-9ae7-6cf32282fd18 service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:44:59.249346 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a53e7d-219e-422f-9ae7-6cf32282fd18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:44:59.257350 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a53e7d-219e-422f-9ae7-6cf32282fd18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:44:59.258781 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a53e7d-219e-422f-9ae7-6cf32282fd18 service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:44:59.267559 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 989/1978] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:44:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:45:09.173908 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=17 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:09.263544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=40,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:11.832875 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e73ad9-bc11-4b23-b523-5d6f4b365ec8 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:11.834458 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-23e73ad9-bc11-4b23-b523-5d6f4b365ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:11.901916 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e73ad9-bc11-4b23-b523-5d6f4b365ec8 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:11.907353 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968bc6b3-a5db-4dea-b145-992f454948c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:11.908355 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bc6b3-a5db-4dea-b145-992f454948c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:11.908470 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bc6b3-a5db-4dea-b145-992f454948c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:11.908747 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bc6b3-a5db-4dea-b145-992f454948c1 admin admin] PATH_INFO: `/v3/users/7d73f4f2c8934c3a927027a0893055c7` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:11.909919 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968bc6b3-a5db-4dea-b145-992f454948c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d73f4f2c8934c3a927027a0893055c7)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:11.912153 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968bc6b3-a5db-4dea-b145-992f454948c1 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:11.989496 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-968bc6b3-a5db-4dea-b145-992f454948c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d73f4f2c8934c3a927027a0893055c7'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:11.990561 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-968bc6b3-a5db-4dea-b145-992f454948c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d73f4f2c8934c3a927027a0893055c7'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:11.991199 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-968bc6b3-a5db-4dea-b145-992f454948c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d73f4f2c8934c3a927027a0893055c7'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:12.002903 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-968bc6b3-a5db-4dea-b145-992f454948c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d73f4f2c8934c3a927027a0893055c7'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:12.015916 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-968bc6b3-a5db-4dea-b145-992f454948c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d73f4f2c8934c3a927027a0893055c7'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:12.043395 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 990/1979] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:45:11 2022] DELETE /identity/v3/users/7d73f4f2c8934c3a927027a0893055c7 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:12.047828 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92584c1c-7d30-4cfc-aa1a-31015ced415b None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:12.048370 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92584c1c-7d30-4cfc-aa1a-31015ced415b None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:12.048598 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92584c1c-7d30-4cfc-aa1a-31015ced415b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:12.068210 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-92584c1c-7d30-4cfc-aa1a-31015ced415b None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:12.069463 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-92584c1c-7d30-4cfc-aa1a-31015ced415b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:12.146683 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 990/1980] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:45:12.162983 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968bc6b3-a5db-4dea-b145-992f454948c1 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:12.162983 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-968bc6b3-a5db-4dea-b145-992f454948c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:12.252686 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968bc6b3-a5db-4dea-b145-992f454948c1 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:12.256312 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c12ec-55d9-4cdc-bb77-089739e23960 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:12.256933 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c12ec-55d9-4cdc-bb77-089739e23960 service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:12.257213 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c12ec-55d9-4cdc-bb77-089739e23960 service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:12.257492 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c12ec-55d9-4cdc-bb77-089739e23960 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:12.267641 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4c12ec-55d9-4cdc-bb77-089739e23960 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:12.269098 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4c12ec-55d9-4cdc-bb77-089739e23960 service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:12.286380 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 991/1981] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:45:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:45:12.633162 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92584c1c-7d30-4cfc-aa1a-31015ced415b None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:12.635332 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-92584c1c-7d30-4cfc-aa1a-31015ced415b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:12.661062 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92584c1c-7d30-4cfc-aa1a-31015ced415b None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:12.672050 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3544d-1f9c-48fa-b386-f5249127c736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:12.673210 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3544d-1f9c-48fa-b386-f5249127c736 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:12.673710 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3544d-1f9c-48fa-b386-f5249127c736 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:12.674183 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3544d-1f9c-48fa-b386-f5249127c736 admin admin] PATH_INFO: `/v3/projects/41b712b7f7a44dad8f2ab48460260f13` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:12.675553 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c3544d-1f9c-48fa-b386-f5249127c736 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41b712b7f7a44dad8f2ab48460260f13)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:12.677713 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c3544d-1f9c-48fa-b386-f5249127c736 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:12.732461 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c12ec-55d9-4cdc-bb77-089739e23960 service neutron] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:12.734002 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-8d4c12ec-55d9-4cdc-bb77-089739e23960 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:12.739005 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-f5c3544d-1f9c-48fa-b386-f5249127c736 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41b712b7f7a44dad8f2ab48460260f13', 'request_id': 'req-f5c3544d-1f9c-48fa-b386-f5249127c736'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:12.745611 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-f5c3544d-1f9c-48fa-b386-f5249127c736 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41b712b7f7a44dad8f2ab48460260f13', 'request_id': 'req-f5c3544d-1f9c-48fa-b386-f5249127c736'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:12.748610 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c12ec-55d9-4cdc-bb77-089739e23960 service neutron] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:12.754618 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-f5c3544d-1f9c-48fa-b386-f5249127c736 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41b712b7f7a44dad8f2ab48460260f13', 'request_id': 'req-f5c3544d-1f9c-48fa-b386-f5249127c736'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:12.770179 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-f5c3544d-1f9c-48fa-b386-f5249127c736 admin admin] The token cache is being invalidate because project 41b712b7f7a44dad8f2ab48460260f13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:45:12.770521 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-f5c3544d-1f9c-48fa-b386-f5249127c736 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:12.771128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-f5c3544d-1f9c-48fa-b386-f5249127c736 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:12.772679 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 991/1982] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:45:12 2022] DELETE /identity/v3/projects/41b712b7f7a44dad8f2ab48460260f13 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:12.776079 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:12.776645 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3544d-1f9c-48fa-b386-f5249127c736 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:12.778043 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-f5c3544d-1f9c-48fa-b386-f5249127c736 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:12.836318 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9bdbf5-bd35-4eed-9876-92fb22efc6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:12.837985 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9bdbf5-bd35-4eed-9876-92fb22efc6c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:12.838481 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9bdbf5-bd35-4eed-9876-92fb22efc6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:12.839225 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9bdbf5-bd35-4eed-9876-92fb22efc6c3 admin admin] PATH_INFO: `/v3/users/f23cd844f473475d8762bd155345f44d` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:12.840640 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9bdbf5-bd35-4eed-9876-92fb22efc6c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f23cd844f473475d8762bd155345f44d)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:12.843839 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9bdbf5-bd35-4eed-9876-92fb22efc6c3 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:12.880024 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3544d-1f9c-48fa-b386-f5249127c736 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:12.884963 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839ca94-58a0-45d7-b5a5-07e8d1bd6154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:12.885750 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839ca94-58a0-45d7-b5a5-07e8d1bd6154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:12.886036 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839ca94-58a0-45d7-b5a5-07e8d1bd6154 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:12.886313 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839ca94-58a0-45d7-b5a5-07e8d1bd6154 admin admin] PATH_INFO: `/v3/users/e8ecaa094aba4624b4408df49341c217` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:12.888298 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8839ca94-58a0-45d7-b5a5-07e8d1bd6154 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8ecaa094aba4624b4408df49341c217)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:12.890304 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8839ca94-58a0-45d7-b5a5-07e8d1bd6154 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:12.928196 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:12.931454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-2b9bdbf5-bd35-4eed-9876-92fb22efc6c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f23cd844f473475d8762bd155345f44d'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:12.934437 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-2b9bdbf5-bd35-4eed-9876-92fb22efc6c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f23cd844f473475d8762bd155345f44d'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:12.935311 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-2b9bdbf5-bd35-4eed-9876-92fb22efc6c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f23cd844f473475d8762bd155345f44d'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:12.951381 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-2b9bdbf5-bd35-4eed-9876-92fb22efc6c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f23cd844f473475d8762bd155345f44d'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:12.957861 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-8839ca94-58a0-45d7-b5a5-07e8d1bd6154 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8ecaa094aba4624b4408df49341c217'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:12.958843 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-8839ca94-58a0-45d7-b5a5-07e8d1bd6154 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8ecaa094aba4624b4408df49341c217'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:12.961905 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-2b9bdbf5-bd35-4eed-9876-92fb22efc6c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f23cd844f473475d8762bd155345f44d'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:12.974538 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-8839ca94-58a0-45d7-b5a5-07e8d1bd6154 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8ecaa094aba4624b4408df49341c217'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:12.982996 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 992/1983] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:45:12 2022] DELETE /identity/v3/users/f23cd844f473475d8762bd155345f44d => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:12.983760 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-8839ca94-58a0-45d7-b5a5-07e8d1bd6154 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8ecaa094aba4624b4408df49341c217'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:12.984600 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-8839ca94-58a0-45d7-b5a5-07e8d1bd6154 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8ecaa094aba4624b4408df49341c217'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:12.987082 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05bf4cb-29ca-42a1-8b15-2979d9fe93e4 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:12.987381 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05bf4cb-29ca-42a1-8b15-2979d9fe93e4 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:12.987663 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05bf4cb-29ca-42a1-8b15-2979d9fe93e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:13.003273 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 992/1984] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:45:12 2022] DELETE /identity/v3/users/e8ecaa094aba4624b4408df49341c217 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:45:13.010793 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-c05bf4cb-29ca-42a1-8b15-2979d9fe93e4 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:13.012365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-c05bf4cb-29ca-42a1-8b15-2979d9fe93e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:13.108225 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 993/1985] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:45:13.122993 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839ca94-58a0-45d7-b5a5-07e8d1bd6154 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:13.130352 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-8839ca94-58a0-45d7-b5a5-07e8d1bd6154 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:13.230042 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839ca94-58a0-45d7-b5a5-07e8d1bd6154 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:13.235489 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a9aca0-7913-4e6b-8209-bb367c024dcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:13.236640 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9aca0-7913-4e6b-8209-bb367c024dcf service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:13.236954 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9aca0-7913-4e6b-8209-bb367c024dcf service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:13.239107 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9aca0-7913-4e6b-8209-bb367c024dcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:13.250970 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a9aca0-7913-4e6b-8209-bb367c024dcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:13.252554 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a9aca0-7913-4e6b-8209-bb367c024dcf service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:13.264654 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 993/1986] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:45:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:45:13.610067 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05bf4cb-29ca-42a1-8b15-2979d9fe93e4 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:13.612431 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-c05bf4cb-29ca-42a1-8b15-2979d9fe93e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:13.641892 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05bf4cb-29ca-42a1-8b15-2979d9fe93e4 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:13.672040 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad687d8a-f718-4d52-a9d0-4cf53cf83666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:13.672883 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad687d8a-f718-4d52-a9d0-4cf53cf83666 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:13.673675 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad687d8a-f718-4d52-a9d0-4cf53cf83666 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:13.673675 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad687d8a-f718-4d52-a9d0-4cf53cf83666 admin admin] PATH_INFO: `/v3/projects/ea06144937534994908fb8349b454918` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:13.674831 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad687d8a-f718-4d52-a9d0-4cf53cf83666 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea06144937534994908fb8349b454918)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:13.679465 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad687d8a-f718-4d52-a9d0-4cf53cf83666 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:13.789739 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:13.804620 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-ad687d8a-f718-4d52-a9d0-4cf53cf83666 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea06144937534994908fb8349b454918', 'request_id': 'req-ad687d8a-f718-4d52-a9d0-4cf53cf83666'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:13.816473 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-ad687d8a-f718-4d52-a9d0-4cf53cf83666 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea06144937534994908fb8349b454918', 'request_id': 'req-ad687d8a-f718-4d52-a9d0-4cf53cf83666'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:13.828049 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-ad687d8a-f718-4d52-a9d0-4cf53cf83666 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea06144937534994908fb8349b454918', 'request_id': 'req-ad687d8a-f718-4d52-a9d0-4cf53cf83666'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:13.841597 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a9aca0-7913-4e6b-8209-bb367c024dcf service neutron] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:13.843595 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-f5a9aca0-7913-4e6b-8209-bb367c024dcf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:13.866121 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-ad687d8a-f718-4d52-a9d0-4cf53cf83666 admin admin] The token cache is being invalidate because project ea06144937534994908fb8349b454918 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:45:13.867289 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-ad687d8a-f718-4d52-a9d0-4cf53cf83666 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:13.869714 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-ad687d8a-f718-4d52-a9d0-4cf53cf83666 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:13.872933 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 994/1987] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:45:13 2022] DELETE /identity/v3/projects/ea06144937534994908fb8349b454918 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:13.877885 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b5c91-aa34-47e5-9403-07dc7a02615f None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:13.878464 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b5c91-aa34-47e5-9403-07dc7a02615f None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:13.878974 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b5c91-aa34-47e5-9403-07dc7a02615f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:13.900885 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a9aca0-7913-4e6b-8209-bb367c024dcf service neutron] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:13.910047 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-c55b5c91-aa34-47e5-9403-07dc7a02615f None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:13.913317 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-c55b5c91-aa34-47e5-9403-07dc7a02615f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:13.991722 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef60fb24-3bdf-411a-8617-08d7fc1aaeca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:13.993470 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60fb24-3bdf-411a-8617-08d7fc1aaeca admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:13.993970 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60fb24-3bdf-411a-8617-08d7fc1aaeca admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:13.994515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60fb24-3bdf-411a-8617-08d7fc1aaeca admin admin] PATH_INFO: `/v3/projects/e8f882aac0434f49af89ed8c566c057a` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:13.996826 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef60fb24-3bdf-411a-8617-08d7fc1aaeca admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8f882aac0434f49af89ed8c566c057a)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:13.999323 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef60fb24-3bdf-411a-8617-08d7fc1aaeca admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:14.008199 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 995/1988] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:45:14.008199 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55b5c91-aa34-47e5-9403-07dc7a02615f None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:14.009932 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-c55b5c91-aa34-47e5-9403-07dc7a02615f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:14.025380 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55b5c91-aa34-47e5-9403-07dc7a02615f None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:14.030864 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d4e3a-ce66-4af8-a127-b6060328634a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:14.031527 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d4e3a-ce66-4af8-a127-b6060328634a admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:14.031809 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d4e3a-ce66-4af8-a127-b6060328634a admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:14.032128 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d4e3a-ce66-4af8-a127-b6060328634a admin admin] PATH_INFO: `/v3/users/d9db8c44b6c64ff7b54159278400063c` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:14.033339 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0d4e3a-ce66-4af8-a127-b6060328634a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9db8c44b6c64ff7b54159278400063c)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:14.035597 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0d4e3a-ce66-4af8-a127-b6060328634a admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:14.075629 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:14.078304 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-ef60fb24-3bdf-411a-8617-08d7fc1aaeca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8f882aac0434f49af89ed8c566c057a', 'request_id': 'req-ef60fb24-3bdf-411a-8617-08d7fc1aaeca'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:14.092308 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-ef60fb24-3bdf-411a-8617-08d7fc1aaeca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8f882aac0434f49af89ed8c566c057a', 'request_id': 'req-ef60fb24-3bdf-411a-8617-08d7fc1aaeca'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:14.103785 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-7e0d4e3a-ce66-4af8-a127-b6060328634a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9db8c44b6c64ff7b54159278400063c'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:14.104732 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-7e0d4e3a-ce66-4af8-a127-b6060328634a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9db8c44b6c64ff7b54159278400063c'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:14.105497 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-7e0d4e3a-ce66-4af8-a127-b6060328634a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9db8c44b6c64ff7b54159278400063c'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:14.106946 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-ef60fb24-3bdf-411a-8617-08d7fc1aaeca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8f882aac0434f49af89ed8c566c057a', 'request_id': 'req-ef60fb24-3bdf-411a-8617-08d7fc1aaeca'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:14.116032 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-7e0d4e3a-ce66-4af8-a127-b6060328634a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9db8c44b6c64ff7b54159278400063c'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:14.125480 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-ef60fb24-3bdf-411a-8617-08d7fc1aaeca admin admin] The token cache is being invalidate because project e8f882aac0434f49af89ed8c566c057a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:45:14.126097 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-ef60fb24-3bdf-411a-8617-08d7fc1aaeca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:14.127079 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-ef60fb24-3bdf-411a-8617-08d7fc1aaeca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:14.129423 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 994/1989] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:45:13 2022] DELETE /identity/v3/projects/e8f882aac0434f49af89ed8c566c057a => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:14.131320 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd2f860-987b-4549-ba6a-5281509dd3a3 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:14.131506 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd2f860-987b-4549-ba6a-5281509dd3a3 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:14.131747 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd2f860-987b-4549-ba6a-5281509dd3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:14.132207 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-7e0d4e3a-ce66-4af8-a127-b6060328634a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9db8c44b6c64ff7b54159278400063c'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:14.156104 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-2fd2f860-987b-4549-ba6a-5281509dd3a3 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:14.159799 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-2fd2f860-987b-4549-ba6a-5281509dd3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:14.178274 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 996/1990] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:45:14 2022] DELETE /identity/v3/users/d9db8c44b6c64ff7b54159278400063c => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:14.179910 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d4e3a-ce66-4af8-a127-b6060328634a admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:14.182331 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-7e0d4e3a-ce66-4af8-a127-b6060328634a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:14.256467 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:14.292383 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 995/1991] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:45:14.299267 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d42be3-7f6c-4438-b9e3-4e87ffdc7153 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:14.299581 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d42be3-7f6c-4438-b9e3-4e87ffdc7153 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:14.299856 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d42be3-7f6c-4438-b9e3-4e87ffdc7153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:14.317783 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d4e3a-ce66-4af8-a127-b6060328634a admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:14.324233 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636cc8b-6c93-4add-b3b0-7e724ce4ad09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:14.324906 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636cc8b-6c93-4add-b3b0-7e724ce4ad09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:14.325186 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636cc8b-6c93-4add-b3b0-7e724ce4ad09 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:14.325460 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636cc8b-6c93-4add-b3b0-7e724ce4ad09 admin admin] PATH_INFO: `/v3/users/e9db338e2d544688840ac4a7f3114304` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:14.326685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3636cc8b-6c93-4add-b3b0-7e724ce4ad09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9db338e2d544688840ac4a7f3114304)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:14.329290 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3636cc8b-6c93-4add-b3b0-7e724ce4ad09 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:14.339709 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-b9d42be3-7f6c-4438-b9e3-4e87ffdc7153 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:14.343072 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-b9d42be3-7f6c-4438-b9e3-4e87ffdc7153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:14.363839 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 996/1992] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:45:14.364972 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d42be3-7f6c-4438-b9e3-4e87ffdc7153 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:14.380624 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d42be3-7f6c-4438-b9e3-4e87ffdc7153 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:14.387293 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8024907d-4b65-47c3-bfcd-a4bd243565d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:14.388188 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024907d-4b65-47c3-bfcd-a4bd243565d3 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:14.388771 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024907d-4b65-47c3-bfcd-a4bd243565d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:14.389869 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024907d-4b65-47c3-bfcd-a4bd243565d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:14.391305 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8024907d-4b65-47c3-bfcd-a4bd243565d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:45:14.391690 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8024907d-4b65-47c3-bfcd-a4bd243565d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:14.393741 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8024907d-4b65-47c3-bfcd-a4bd243565d3 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:14.430653 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 997/1993] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:45:14.431554 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8024907d-4b65-47c3-bfcd-a4bd243565d3 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:14.434162 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-8024907d-4b65-47c3-bfcd-a4bd243565d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:14.442830 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-3636cc8b-6c93-4add-b3b0-7e724ce4ad09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9db338e2d544688840ac4a7f3114304'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:14.443602 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-3636cc8b-6c93-4add-b3b0-7e724ce4ad09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9db338e2d544688840ac4a7f3114304'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:14.446501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-3636cc8b-6c93-4add-b3b0-7e724ce4ad09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9db338e2d544688840ac4a7f3114304'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:14.456855 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-3636cc8b-6c93-4add-b3b0-7e724ce4ad09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9db338e2d544688840ac4a7f3114304'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:14.469894 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-3636cc8b-6c93-4add-b3b0-7e724ce4ad09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9db338e2d544688840ac4a7f3114304'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:14.499547 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 997/1994] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:45:14 2022] DELETE /identity/v3/users/e9db338e2d544688840ac4a7f3114304 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:14.500451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636cc8b-6c93-4add-b3b0-7e724ce4ad09 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:14.505262 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-3636cc8b-6c93-4add-b3b0-7e724ce4ad09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:14.513659 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8024907d-4b65-47c3-bfcd-a4bd243565d3 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:14.580774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0432039b-e0a0-4f3d-91cc-9252e4f548bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:14.584001 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0432039b-e0a0-4f3d-91cc-9252e4f548bb service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:14.585869 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0432039b-e0a0-4f3d-91cc-9252e4f548bb service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:14.586286 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0432039b-e0a0-4f3d-91cc-9252e4f548bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:14.590743 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-0432039b-e0a0-4f3d-91cc-9252e4f548bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:14.629152 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:14.630192 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636cc8b-6c93-4add-b3b0-7e724ce4ad09 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:14.649324 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb56eb-0e69-4fb1-9898-c1f01de338e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:14.649872 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb56eb-0e69-4fb1-9898-c1f01de338e7 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:14.650151 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb56eb-0e69-4fb1-9898-c1f01de338e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:14.650421 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb56eb-0e69-4fb1-9898-c1f01de338e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:14.653304 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abb56eb-0e69-4fb1-9898-c1f01de338e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:14.655282 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abb56eb-0e69-4fb1-9898-c1f01de338e7 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:14.716480 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 998/1995] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:45:14 2022] POST /identity/v3/projects => generated 367 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:45:14.719469 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0432039b-e0a0-4f3d-91cc-9252e4f548bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:14.720800 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb56eb-0e69-4fb1-9898-c1f01de338e7 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:14.724409 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0432039b-e0a0-4f3d-91cc-9252e4f548bb service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:14.756795 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 998/1996] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:45:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:45:14.777601 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:14.806023 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb56eb-0e69-4fb1-9898-c1f01de338e7 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:14.813529 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4da913d-3b53-4066-83e5-949b6e1a64de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:14.814523 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da913d-3b53-4066-83e5-949b6e1a64de admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:14.814965 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da913d-3b53-4066-83e5-949b6e1a64de admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:14.815402 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da913d-3b53-4066-83e5-949b6e1a64de admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:14.816754 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4da913d-3b53-4066-83e5-949b6e1a64de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:14.818761 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4da913d-3b53-4066-83e5-949b6e1a64de admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:14.862688 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 999/1997] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:45:14 2022] POST /identity/v3/users => generated 284 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:45:14.867256 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0432039b-e0a0-4f3d-91cc-9252e4f548bb service neutron] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:14.878680 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0432039b-e0a0-4f3d-91cc-9252e4f548bb service neutron] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:14.884461 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e97cd-7c01-4765-a966-f0a646e7ebaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:14.884925 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e97cd-7c01-4765-a966-f0a646e7ebaf admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:14.885203 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e97cd-7c01-4765-a966-f0a646e7ebaf admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:14.888367 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e97cd-7c01-4765-a966-f0a646e7ebaf admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:14.889143 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727e97cd-7c01-4765-a966-f0a646e7ebaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:14.890880 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727e97cd-7c01-4765-a966-f0a646e7ebaf admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:14.904014 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 999/1998] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:45:14.906913 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4da913d-3b53-4066-83e5-949b6e1a64de admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:14.928341 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4da913d-3b53-4066-83e5-949b6e1a64de admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:14.934206 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b123274-c720-4308-a65f-27668fed2f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:14.934912 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b123274-c720-4308-a65f-27668fed2f85 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:14.935195 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b123274-c720-4308-a65f-27668fed2f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:14.935464 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b123274-c720-4308-a65f-27668fed2f85 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:14.936135 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b123274-c720-4308-a65f-27668fed2f85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:14.937734 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b123274-c720-4308-a65f-27668fed2f85 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:14.949569 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1000/1999] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:45:14.954023 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e97cd-7c01-4765-a966-f0a646e7ebaf admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:14.965177 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e97cd-7c01-4765-a966-f0a646e7ebaf admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:14.971046 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c7f6e-e6a7-4420-a223-0a1b218dbc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:14.971887 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c7f6e-e6a7-4420-a223-0a1b218dbc69 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:14.972098 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c7f6e-e6a7-4420-a223-0a1b218dbc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:14.972371 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c7f6e-e6a7-4420-a223-0a1b218dbc69 admin admin] PATH_INFO: `/v3/projects/b9c2cad2bb2149698dabf1c88508a9b0/users/56bbec5df7354cc1bed911a1e4575c29/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:14.973647 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b123274-c720-4308-a65f-27668fed2f85 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:14.977761 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-3b123274-c720-4308-a65f-27668fed2f85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:14.994612 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b123274-c720-4308-a65f-27668fed2f85 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:14.994972 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c7f6e-e6a7-4420-a223-0a1b218dbc69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9c2cad2bb2149698dabf1c88508a9b0, user_id=56bbec5df7354cc1bed911a1e4575c29, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:14.998838 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c7f6e-e6a7-4420-a223-0a1b218dbc69 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:15.014959 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f82da06-9e5f-43ba-9ba3-31182e1efd88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:15.015658 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82da06-9e5f-43ba-9ba3-31182e1efd88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:15.015936 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82da06-9e5f-43ba-9ba3-31182e1efd88 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:15.016228 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82da06-9e5f-43ba-9ba3-31182e1efd88 admin admin] PATH_INFO: `/v3/projects/1a6272d49e994821b53a12be3fd7497b` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:15.018815 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1000/2000] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:45:14 2022] PUT /identity/v3/projects/b9c2cad2bb2149698dabf1c88508a9b0/users/56bbec5df7354cc1bed911a1e4575c29/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:45:15.021326 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f82da06-9e5f-43ba-9ba3-31182e1efd88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a6272d49e994821b53a12be3fd7497b)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:15.023261 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f82da06-9e5f-43ba-9ba3-31182e1efd88 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:15.031012 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcef32-7012-424d-9aed-cfba93e8cc7a None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:15.031012 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcef32-7012-424d-9aed-cfba93e8cc7a None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:15.031257 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcef32-7012-424d-9aed-cfba93e8cc7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:15.063411 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-e6bcef32-7012-424d-9aed-cfba93e8cc7a None None] MFA Rules not processed for user `56bbec5df7354cc1bed911a1e4575c29`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:15.064718 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-e6bcef32-7012-424d-9aed-cfba93e8cc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:15.088925 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-6f82da06-9e5f-43ba-9ba3-31182e1efd88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a6272d49e994821b53a12be3fd7497b', 'request_id': 'req-6f82da06-9e5f-43ba-9ba3-31182e1efd88'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:15.098784 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-6f82da06-9e5f-43ba-9ba3-31182e1efd88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a6272d49e994821b53a12be3fd7497b', 'request_id': 'req-6f82da06-9e5f-43ba-9ba3-31182e1efd88'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:15.111329 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-6f82da06-9e5f-43ba-9ba3-31182e1efd88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a6272d49e994821b53a12be3fd7497b', 'request_id': 'req-6f82da06-9e5f-43ba-9ba3-31182e1efd88'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:15.124982 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-6f82da06-9e5f-43ba-9ba3-31182e1efd88 admin admin] The token cache is being invalidate because project 1a6272d49e994821b53a12be3fd7497b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:45:15.125271 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-6f82da06-9e5f-43ba-9ba3-31182e1efd88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:15.126547 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-6f82da06-9e5f-43ba-9ba3-31182e1efd88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:15.128222 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1001/2001] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:45:14 2022] DELETE /identity/v3/projects/1a6272d49e994821b53a12be3fd7497b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:15.134261 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce71676-1400-4536-87f2-71bce35d3bba None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:15.134512 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce71676-1400-4536-87f2-71bce35d3bba None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:15.136742 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce71676-1400-4536-87f2-71bce35d3bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:15.149690 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1001/2002] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:45:15.154145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bcef32-7012-424d-9aed-cfba93e8cc7a None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:15.158238 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-e6bcef32-7012-424d-9aed-cfba93e8cc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:15.158479 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-8ce71676-1400-4536-87f2-71bce35d3bba None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:15.158479 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-8ce71676-1400-4536-87f2-71bce35d3bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:15.236228 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1002/2003] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:45:15.237302 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce71676-1400-4536-87f2-71bce35d3bba None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:15.245178 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-8ce71676-1400-4536-87f2-71bce35d3bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:15.257388 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bcef32-7012-424d-9aed-cfba93e8cc7a None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:15.257747 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce71676-1400-4536-87f2-71bce35d3bba None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:15.261917 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11028075-c3c1-45f7-9264-546bc790d531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:15.262298 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0008e-42b1-4b50-9f8c-44205b4bcc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:15.262544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11028075-c3c1-45f7-9264-546bc790d531 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:15.262823 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11028075-c3c1-45f7-9264-546bc790d531 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:15.262931 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0008e-42b1-4b50-9f8c-44205b4bcc3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:15.263079 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11028075-c3c1-45f7-9264-546bc790d531 admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:15.263181 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0008e-42b1-4b50-9f8c-44205b4bcc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:15.263442 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0008e-42b1-4b50-9f8c-44205b4bcc3b admin admin] PATH_INFO: `/v3/projects/b391ac03862c41279b782a53bf0c7310` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:15.263696 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11028075-c3c1-45f7-9264-546bc790d531 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:15.264607 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0008e-42b1-4b50-9f8c-44205b4bcc3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b391ac03862c41279b782a53bf0c7310)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:15.265637 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11028075-c3c1-45f7-9264-546bc790d531 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:15.266537 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0008e-42b1-4b50-9f8c-44205b4bcc3b admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:15.297024 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1002/2004] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:45:15 2022] POST /identity/v3/projects => generated 367 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:45:15.298517 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf421d8-1546-42ab-8bf6-020f2071be18 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:15.298835 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf421d8-1546-42ab-8bf6-020f2071be18 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:15.299163 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf421d8-1546-42ab-8bf6-020f2071be18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:15.307073 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:15.312195 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:15.321634 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-cbf421d8-1546-42ab-8bf6-020f2071be18 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:15.322049 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-17f0008e-42b1-4b50-9f8c-44205b4bcc3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b391ac03862c41279b782a53bf0c7310', 'request_id': 'req-17f0008e-42b1-4b50-9f8c-44205b4bcc3b'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:15.322825 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-cbf421d8-1546-42ab-8bf6-020f2071be18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:15.327872 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-17f0008e-42b1-4b50-9f8c-44205b4bcc3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b391ac03862c41279b782a53bf0c7310', 'request_id': 'req-17f0008e-42b1-4b50-9f8c-44205b4bcc3b'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:15.335638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-17f0008e-42b1-4b50-9f8c-44205b4bcc3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b391ac03862c41279b782a53bf0c7310', 'request_id': 'req-17f0008e-42b1-4b50-9f8c-44205b4bcc3b'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:15.352531 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-17f0008e-42b1-4b50-9f8c-44205b4bcc3b admin admin] The token cache is being invalidate because project b391ac03862c41279b782a53bf0c7310 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:45:15.352820 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-17f0008e-42b1-4b50-9f8c-44205b4bcc3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:15.353507 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-17f0008e-42b1-4b50-9f8c-44205b4bcc3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:15.355778 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1003/2005] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:45:15 2022] DELETE /identity/v3/projects/b391ac03862c41279b782a53bf0c7310 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:15.360267 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0008e-42b1-4b50-9f8c-44205b4bcc3b admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:15.361579 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-17f0008e-42b1-4b50-9f8c-44205b4bcc3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:15.391424 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1003/2006] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:45:15.391424 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63715940-ba5b-4fb1-9c63-a4a1f87c3954 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:15.391424 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63715940-ba5b-4fb1-9c63-a4a1f87c3954 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:15.391424 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63715940-ba5b-4fb1-9c63-a4a1f87c3954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:15.412113 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-63715940-ba5b-4fb1-9c63-a4a1f87c3954 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:15.413328 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-63715940-ba5b-4fb1-9c63-a4a1f87c3954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:15.422583 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1004/2007] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:45:15.423449 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63715940-ba5b-4fb1-9c63-a4a1f87c3954 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:15.432209 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0008e-42b1-4b50-9f8c-44205b4bcc3b admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:15.433657 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63715940-ba5b-4fb1-9c63-a4a1f87c3954 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:15.437755 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a81b9-f2c8-4282-858d-31cc903b6ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:15.438288 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4426e62-ad4d-4f60-9471-0ed55eb30712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:15.438404 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a81b9-f2c8-4282-858d-31cc903b6ffc admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:15.438652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a81b9-f2c8-4282-858d-31cc903b6ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:15.438924 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4426e62-ad4d-4f60-9471-0ed55eb30712 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:15.439021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a81b9-f2c8-4282-858d-31cc903b6ffc admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:15.439181 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4426e62-ad4d-4f60-9471-0ed55eb30712 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:15.439664 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4426e62-ad4d-4f60-9471-0ed55eb30712 admin admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:15.440329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4426e62-ad4d-4f60-9471-0ed55eb30712 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:45:15.440637 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4426e62-ad4d-4f60-9471-0ed55eb30712 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:15.440765 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8a81b9-f2c8-4282-858d-31cc903b6ffc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:15.442773 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8a81b9-f2c8-4282-858d-31cc903b6ffc admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:15.442900 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4426e62-ad4d-4f60-9471-0ed55eb30712 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:15.462459 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1005/2008] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:45:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:45:15.463730 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb768a-eb94-4197-be7d-381f4d00cc53 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:15.464044 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb768a-eb94-4197-be7d-381f4d00cc53 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:15.464320 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb768a-eb94-4197-be7d-381f4d00cc53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:15.467117 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1004/2009] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:45:15 2022] POST /identity/v3/users => generated 290 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:45:15.467908 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a81b9-f2c8-4282-858d-31cc903b6ffc admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:15.481206 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a81b9-f2c8-4282-858d-31cc903b6ffc admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:15.486893 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c056c27-7f5c-4a48-903c-ce8acbc728a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:15.488032 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c056c27-7f5c-4a48-903c-ce8acbc728a2 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:15.488221 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c056c27-7f5c-4a48-903c-ce8acbc728a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:15.489157 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c056c27-7f5c-4a48-903c-ce8acbc728a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:15.489801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c056c27-7f5c-4a48-903c-ce8acbc728a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:15.491752 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c056c27-7f5c-4a48-903c-ce8acbc728a2 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:15.491879 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-3fbb768a-eb94-4197-be7d-381f4d00cc53 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:15.493102 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-3fbb768a-eb94-4197-be7d-381f4d00cc53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:15.504601 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1006/2010] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:45:15.506475 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb768a-eb94-4197-be7d-381f4d00cc53 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:15.517365 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb768a-eb94-4197-be7d-381f4d00cc53 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:15.521925 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c4c0f-4bb0-47c7-8b1a-25c77c13d234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:15.522517 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c4c0f-4bb0-47c7-8b1a-25c77c13d234 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:15.522792 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c4c0f-4bb0-47c7-8b1a-25c77c13d234 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:15.524009 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1005/2011] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:45:15 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:45:15.524135 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c4c0f-4bb0-47c7-8b1a-25c77c13d234 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:15.524135 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c4c0f-4bb0-47c7-8b1a-25c77c13d234 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:15.524319 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c056c27-7f5c-4a48-903c-ce8acbc728a2 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:15.525290 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c4c0f-4bb0-47c7-8b1a-25c77c13d234 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:15.537135 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1007/2012] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:45:15.537925 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c4c0f-4bb0-47c7-8b1a-25c77c13d234 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:15.548937 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c4c0f-4bb0-47c7-8b1a-25c77c13d234 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:15.592440 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c056c27-7f5c-4a48-903c-ce8acbc728a2 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:15.597117 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7006b86-aaaa-429f-a538-548da9ba86d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:15.597722 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7006b86-aaaa-429f-a538-548da9ba86d4 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:15.598004 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7006b86-aaaa-429f-a538-548da9ba86d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:15.598280 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7006b86-aaaa-429f-a538-548da9ba86d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:15.598929 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7006b86-aaaa-429f-a538-548da9ba86d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:45:15.599237 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7006b86-aaaa-429f-a538-548da9ba86d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:15.600871 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7006b86-aaaa-429f-a538-548da9ba86d4 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:15.609341 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a625ff-2a62-4b55-bfb8-21fdd07a7e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:15.609965 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a625ff-2a62-4b55-bfb8-21fdd07a7e1a admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:15.610248 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a625ff-2a62-4b55-bfb8-21fdd07a7e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:15.610990 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a625ff-2a62-4b55-bfb8-21fdd07a7e1a admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:15.611926 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a625ff-2a62-4b55-bfb8-21fdd07a7e1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:15.617256 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1006/2013] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:45:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:45:15.617499 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a625ff-2a62-4b55-bfb8-21fdd07a7e1a admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:15.617587 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7006b86-aaaa-429f-a538-548da9ba86d4 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:15.628678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7006b86-aaaa-429f-a538-548da9ba86d4 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:15.634243 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff36ee2-9476-4077-8bdb-235d09cbb127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:15.634972 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff36ee2-9476-4077-8bdb-235d09cbb127 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:15.635256 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff36ee2-9476-4077-8bdb-235d09cbb127 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:15.635549 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff36ee2-9476-4077-8bdb-235d09cbb127 admin admin] PATH_INFO: `/v3/projects/472844468f254c488a96c2707cf34bbf/users/d8658f4423144dec9cfa89c0ae92d866/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:15.639211 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1008/2014] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:45:15 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:45:15.639211 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a625ff-2a62-4b55-bfb8-21fdd07a7e1a admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:15.646504 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a625ff-2a62-4b55-bfb8-21fdd07a7e1a admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:15.651203 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db3a2f-7146-4d9c-8091-1429061ff06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:15.651786 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db3a2f-7146-4d9c-8091-1429061ff06d admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:15.652631 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db3a2f-7146-4d9c-8091-1429061ff06d admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:15.652903 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db3a2f-7146-4d9c-8091-1429061ff06d admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:15.653527 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db3a2f-7146-4d9c-8091-1429061ff06d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:15.653630 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff36ee2-9476-4077-8bdb-235d09cbb127 admin admin] RBAC: Authorizing `identity:create_grant(project_id=472844468f254c488a96c2707cf34bbf, user_id=d8658f4423144dec9cfa89c0ae92d866, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:15.655420 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db3a2f-7146-4d9c-8091-1429061ff06d admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:15.656257 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff36ee2-9476-4077-8bdb-235d09cbb127 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:15.670204 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1007/2015] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:45:15 2022] PUT /identity/v3/projects/472844468f254c488a96c2707cf34bbf/users/d8658f4423144dec9cfa89c0ae92d866/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:15.674402 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff36ee2-9476-4077-8bdb-235d09cbb127 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:15.682973 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=90,keystone:DELETE=4,keystone:INSERT=6 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:15.692144 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1009/2016] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:45:15 2022] POST /identity/v3/projects => generated 399 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:45:15.692144 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db3a2f-7146-4d9c-8091-1429061ff06d admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:15.728287 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:15.753406 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff36ee2-9476-4077-8bdb-235d09cbb127 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:15.758053 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57dae3-9e63-44db-8e69-09f71392cbc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:15.758687 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57dae3-9e63-44db-8e69-09f71392cbc4 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:15.758906 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db3a2f-7146-4d9c-8091-1429061ff06d admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:15.759023 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57dae3-9e63-44db-8e69-09f71392cbc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:15.759219 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57dae3-9e63-44db-8e69-09f71392cbc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:15.759927 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c57dae3-9e63-44db-8e69-09f71392cbc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:15.761522 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c57dae3-9e63-44db-8e69-09f71392cbc4 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:15.763612 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb766fe4-a1ff-4f77-bc97-b9cce12ba29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:15.764243 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb766fe4-a1ff-4f77-bc97-b9cce12ba29c admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:15.764519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb766fe4-a1ff-4f77-bc97-b9cce12ba29c admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:15.764789 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb766fe4-a1ff-4f77-bc97-b9cce12ba29c admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:15.765432 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb766fe4-a1ff-4f77-bc97-b9cce12ba29c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:15.767134 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb766fe4-a1ff-4f77-bc97-b9cce12ba29c admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:15.771416 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1008/2017] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:45:15.772925 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57dae3-9e63-44db-8e69-09f71392cbc4 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:15.778151 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1010/2018] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:45:15.778885 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb766fe4-a1ff-4f77-bc97-b9cce12ba29c admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:15.783374 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57dae3-9e63-44db-8e69-09f71392cbc4 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:15.789261 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb766fe4-a1ff-4f77-bc97-b9cce12ba29c admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:15.790965 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa51a6-5c2c-4b68-aa3e-90fe9cc668cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:15.792091 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa51a6-5c2c-4b68-aa3e-90fe9cc668cd admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:15.792648 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa51a6-5c2c-4b68-aa3e-90fe9cc668cd admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:15.793716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa51a6-5c2c-4b68-aa3e-90fe9cc668cd admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:15.794822 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa51a6-5c2c-4b68-aa3e-90fe9cc668cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:15.794945 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba5c4d-9fd5-4334-b967-0de3f7d6d16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:15.795448 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba5c4d-9fd5-4334-b967-0de3f7d6d16f admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:15.795726 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba5c4d-9fd5-4334-b967-0de3f7d6d16f admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:15.796051 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba5c4d-9fd5-4334-b967-0de3f7d6d16f admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:15.796682 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcba5c4d-9fd5-4334-b967-0de3f7d6d16f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:15.797336 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa51a6-5c2c-4b68-aa3e-90fe9cc668cd admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:15.798245 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcba5c4d-9fd5-4334-b967-0de3f7d6d16f admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:15.808280 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1011/2019] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:45:15.810863 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba5c4d-9fd5-4334-b967-0de3f7d6d16f admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:15.826888 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1009/2020] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:45:15 2022] POST /identity/v3/users => generated 300 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:45:15.826888 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa51a6-5c2c-4b68-aa3e-90fe9cc668cd admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:15.827149 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba5c4d-9fd5-4334-b967-0de3f7d6d16f admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:15.830060 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e0b17-2e98-4606-83be-5561c7f96bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:15.830709 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e0b17-2e98-4606-83be-5561c7f96bdc admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:15.830990 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e0b17-2e98-4606-83be-5561c7f96bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:15.831253 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e0b17-2e98-4606-83be-5561c7f96bdc admin admin] PATH_INFO: `/v3/projects/472844468f254c488a96c2707cf34bbf/users/d8658f4423144dec9cfa89c0ae92d866/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:15.833184 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e0b17-2e98-4606-83be-5561c7f96bdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=472844468f254c488a96c2707cf34bbf, user_id=d8658f4423144dec9cfa89c0ae92d866, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:15.834353 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa51a6-5c2c-4b68-aa3e-90fe9cc668cd admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:15.836553 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e0b17-2e98-4606-83be-5561c7f96bdc admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:15.838874 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2105c6e2-f092-437e-8d4e-30aaa8398fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:15.840435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105c6e2-f092-437e-8d4e-30aaa8398fdb admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:15.840435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105c6e2-f092-437e-8d4e-30aaa8398fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:15.840621 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105c6e2-f092-437e-8d4e-30aaa8398fdb admin admin] PATH_INFO: `/v3/projects/fe1305efdd04422dbc1a297de85f4319/users/62bec362c71848e5b786ba1cde67a776/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:15.848911 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1012/2021] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:45:15 2022] PUT /identity/v3/projects/472844468f254c488a96c2707cf34bbf/users/d8658f4423144dec9cfa89c0ae92d866/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:15.849585 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e0b17-2e98-4606-83be-5561c7f96bdc admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:15.864107 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2105c6e2-f092-437e-8d4e-30aaa8398fdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe1305efdd04422dbc1a297de85f4319, user_id=62bec362c71848e5b786ba1cde67a776, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:15.866777 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2105c6e2-f092-437e-8d4e-30aaa8398fdb admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:15.880144 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1010/2022] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:45:15 2022] PUT /identity/v3/projects/fe1305efdd04422dbc1a297de85f4319/users/62bec362c71848e5b786ba1cde67a776/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:15.890870 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35bb70-2b2e-4822-8455-fe5491a489d6 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:15.890870 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35bb70-2b2e-4822-8455-fe5491a489d6 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:15.890870 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35bb70-2b2e-4822-8455-fe5491a489d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:15.917047 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-8d35bb70-2b2e-4822-8455-fe5491a489d6 None None] MFA Rules not processed for user `d8658f4423144dec9cfa89c0ae92d866`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:15.919040 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-8d35bb70-2b2e-4822-8455-fe5491a489d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:15.924019 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e0b17-2e98-4606-83be-5561c7f96bdc admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:15.987263 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f6c15-3e14-4f9f-97b4-81510499352e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:15.988308 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f6c15-3e14-4f9f-97b4-81510499352e admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:15.988589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f6c15-3e14-4f9f-97b4-81510499352e admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:15.988869 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f6c15-3e14-4f9f-97b4-81510499352e admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:15.989518 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0f6c15-3e14-4f9f-97b4-81510499352e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:15.991127 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0f6c15-3e14-4f9f-97b4-81510499352e admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:15.997697 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1011/2023] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:15 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:45:15.999006 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564fd2f0-56c5-4d37-8698-b8a386247b62 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:15.999313 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564fd2f0-56c5-4d37-8698-b8a386247b62 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:15.999554 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564fd2f0-56c5-4d37-8698-b8a386247b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:16.004654 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1013/2024] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:45:16.008770 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f6c15-3e14-4f9f-97b4-81510499352e admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:16.021443 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f6c15-3e14-4f9f-97b4-81510499352e admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:16.027661 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca40d5a-59ac-416b-9e4e-06d456af458e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:16.028302 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca40d5a-59ac-416b-9e4e-06d456af458e admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:16.028608 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca40d5a-59ac-416b-9e4e-06d456af458e admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:16.028845 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca40d5a-59ac-416b-9e4e-06d456af458e admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:16.030111 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca40d5a-59ac-416b-9e4e-06d456af458e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:16.030239 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-564fd2f0-56c5-4d37-8698-b8a386247b62 None None] MFA Rules not processed for user `62bec362c71848e5b786ba1cde67a776`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:16.031133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-564fd2f0-56c5-4d37-8698-b8a386247b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:16.031236 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca40d5a-59ac-416b-9e4e-06d456af458e admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:16.044241 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1014/2025] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:45:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:45:16.045113 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca40d5a-59ac-416b-9e4e-06d456af458e admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:16.047369 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-9ca40d5a-59ac-416b-9e4e-06d456af458e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:16.070813 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:16.094103 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:16.110274 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca40d5a-59ac-416b-9e4e-06d456af458e admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:16.112291 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1012/2026] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:45:16.113101 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564fd2f0-56c5-4d37-8698-b8a386247b62 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:16.114073 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c3543f-7a23-4ca8-aeb7-43ca85a03357 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:16.114681 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3543f-7a23-4ca8-aeb7-43ca85a03357 service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:16.114969 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3543f-7a23-4ca8-aeb7-43ca85a03357 service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:16.115237 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3543f-7a23-4ca8-aeb7-43ca85a03357 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:16.123713 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564fd2f0-56c5-4d37-8698-b8a386247b62 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:16.124987 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c3543f-7a23-4ca8-aeb7-43ca85a03357 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:16.126426 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c3543f-7a23-4ca8-aeb7-43ca85a03357 service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:16.128549 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bf9650-58a7-4343-a79b-9ec34934bc68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:16.129194 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf9650-58a7-4343-a79b-9ec34934bc68 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:16.129480 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf9650-58a7-4343-a79b-9ec34934bc68 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:16.129757 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf9650-58a7-4343-a79b-9ec34934bc68 admin admin] PATH_INFO: `/v3/projects/ca5162376893424183ab35bad8fddac3/users/9f1cab1669f74ce4a5e7bda7acddf175/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:16.135823 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1015/2027] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:45:16 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:45:16.137310 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c3543f-7a23-4ca8-aeb7-43ca85a03357 service cinder] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:16.145902 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bf9650-58a7-4343-a79b-9ec34934bc68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca5162376893424183ab35bad8fddac3, user_id=9f1cab1669f74ce4a5e7bda7acddf175, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:16.149200 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bf9650-58a7-4343-a79b-9ec34934bc68 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:16.151398 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c3543f-7a23-4ca8-aeb7-43ca85a03357 service cinder] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:16.156655 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcc880a-2442-490b-b12f-620153868dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:16.157989 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc880a-2442-490b-b12f-620153868dac admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:16.157989 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc880a-2442-490b-b12f-620153868dac admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:16.157989 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc880a-2442-490b-b12f-620153868dac admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:16.158545 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcc880a-2442-490b-b12f-620153868dac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:16.161295 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcc880a-2442-490b-b12f-620153868dac admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:16.168021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1013/2028] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:45:16 2022] PUT /identity/v3/projects/ca5162376893424183ab35bad8fddac3/users/9f1cab1669f74ce4a5e7bda7acddf175/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:16.178799 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf028d-98dc-4131-9883-3132ef72a790 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:16.179096 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf028d-98dc-4131-9883-3132ef72a790 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:16.179448 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf028d-98dc-4131-9883-3132ef72a790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:16.195425 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1016/2029] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:45:16 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:45:16.196539 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcc880a-2442-490b-b12f-620153868dac admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:16.214121 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-cdaf028d-98dc-4131-9883-3132ef72a790 None None] MFA Rules not processed for user `9f1cab1669f74ce4a5e7bda7acddf175`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:16.215358 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-cdaf028d-98dc-4131-9883-3132ef72a790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:16.274386 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcc880a-2442-490b-b12f-620153868dac admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:16.280762 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b4d26-c810-4547-9576-93f72c72b66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:16.281193 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b4d26-c810-4547-9576-93f72c72b66a admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:16.281713 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b4d26-c810-4547-9576-93f72c72b66a admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:16.281838 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b4d26-c810-4547-9576-93f72c72b66a admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:16.282429 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b4d26-c810-4547-9576-93f72c72b66a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:16.284659 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b4d26-c810-4547-9576-93f72c72b66a admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:16.300403 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1014/2030] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:16 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:45:16.301127 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf028d-98dc-4131-9883-3132ef72a790 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:16.312215 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1017/2031] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:45:16 2022] POST /identity/v3/users => generated 299 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:45:16.313259 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b4d26-c810-4547-9576-93f72c72b66a admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:16.315188 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-c60b4d26-c810-4547-9576-93f72c72b66a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:16.369412 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf028d-98dc-4131-9883-3132ef72a790 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:16.373899 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9503f98-558e-408d-b7ea-436cbed66617 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:16.374062 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b4d26-c810-4547-9576-93f72c72b66a admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:16.374662 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9503f98-558e-408d-b7ea-436cbed66617 service cinder] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:16.375019 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9503f98-558e-408d-b7ea-436cbed66617 service cinder] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:16.375296 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9503f98-558e-408d-b7ea-436cbed66617 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:16.377180 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-c9503f98-558e-408d-b7ea-436cbed66617 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:16.377731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362df26d-1f2d-470e-b164-407e7d0afb1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:16.378333 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362df26d-1f2d-470e-b164-407e7d0afb1f service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:16.378609 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362df26d-1f2d-470e-b164-407e7d0afb1f service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:16.378879 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362df26d-1f2d-470e-b164-407e7d0afb1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:16.388051 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362df26d-1f2d-470e-b164-407e7d0afb1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:16.389818 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362df26d-1f2d-470e-b164-407e7d0afb1f service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:16.400286 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1018/2032] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:45:16 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:45:16.401137 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362df26d-1f2d-470e-b164-407e7d0afb1f service cinder] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:16.411795 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362df26d-1f2d-470e-b164-407e7d0afb1f service cinder] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:16.417834 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28884fbc-d3e3-4c0c-82e6-60bc9ef49077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:16.418425 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28884fbc-d3e3-4c0c-82e6-60bc9ef49077 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:16.418704 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28884fbc-d3e3-4c0c-82e6-60bc9ef49077 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:16.418973 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28884fbc-d3e3-4c0c-82e6-60bc9ef49077 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:16.419615 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28884fbc-d3e3-4c0c-82e6-60bc9ef49077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:16.421250 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28884fbc-d3e3-4c0c-82e6-60bc9ef49077 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:16.432443 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1019/2033] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:45:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:45:16.436845 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28884fbc-d3e3-4c0c-82e6-60bc9ef49077 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:16.455955 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:16.458447 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28884fbc-d3e3-4c0c-82e6-60bc9ef49077 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:16.463990 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428d234-7d2d-4f10-ae34-ff3b1a67fb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:16.464955 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428d234-7d2d-4f10-ae34-ff3b1a67fb60 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:16.465232 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428d234-7d2d-4f10-ae34-ff3b1a67fb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:16.465502 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9503f98-558e-408d-b7ea-436cbed66617 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:16.465623 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428d234-7d2d-4f10-ae34-ff3b1a67fb60 admin admin] PATH_INFO: `/v3/projects/95a29dab38fa4b0db6a70390b8a03425/users/0777c39686cc4071bd5bf9325dd7e571/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:16.467052 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9503f98-558e-408d-b7ea-436cbed66617 service cinder] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:16.476377 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1015/2034] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:45:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:45:16.483356 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428d234-7d2d-4f10-ae34-ff3b1a67fb60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95a29dab38fa4b0db6a70390b8a03425, user_id=0777c39686cc4071bd5bf9325dd7e571, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:16.486149 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428d234-7d2d-4f10-ae34-ff3b1a67fb60 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:16.505929 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1020/2035] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:45:16 2022] PUT /identity/v3/projects/95a29dab38fa4b0db6a70390b8a03425/users/0777c39686cc4071bd5bf9325dd7e571/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:16.509726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9503f98-558e-408d-b7ea-436cbed66617 service cinder] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:16.523988 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9503f98-558e-408d-b7ea-436cbed66617 service cinder] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:16.593347 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea591a71-3d36-4119-b7a7-48621c3d9b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:16.595261 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea591a71-3d36-4119-b7a7-48621c3d9b28 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:16.595261 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea591a71-3d36-4119-b7a7-48621c3d9b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:16.595261 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea591a71-3d36-4119-b7a7-48621c3d9b28 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:16.595731 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea591a71-3d36-4119-b7a7-48621c3d9b28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:16.597390 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea591a71-3d36-4119-b7a7-48621c3d9b28 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:16.611431 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1016/2036] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:45:16 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:45:16.616830 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428d234-7d2d-4f10-ae34-ff3b1a67fb60 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:16.630629 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428d234-7d2d-4f10-ae34-ff3b1a67fb60 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:16.636499 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7e51b-afde-400b-bbea-3b577c024e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:16.637375 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7e51b-afde-400b-bbea-3b577c024e7c admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:16.637726 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7e51b-afde-400b-bbea-3b577c024e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:16.638072 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7e51b-afde-400b-bbea-3b577c024e7c admin admin] PATH_INFO: `/v3/projects/95a29dab38fa4b0db6a70390b8a03425/users/0777c39686cc4071bd5bf9325dd7e571/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:16.641514 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb7e51b-afde-400b-bbea-3b577c024e7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=95a29dab38fa4b0db6a70390b8a03425, user_id=0777c39686cc4071bd5bf9325dd7e571, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:16.644768 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb7e51b-afde-400b-bbea-3b577c024e7c admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:16.660563 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1021/2037] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:45:16 2022] PUT /identity/v3/projects/95a29dab38fa4b0db6a70390b8a03425/users/0777c39686cc4071bd5bf9325dd7e571/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:16.677129 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8931b3-cd73-49f6-a448-450df3c26c66 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:16.677426 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8931b3-cd73-49f6-a448-450df3c26c66 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:16.677701 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8931b3-cd73-49f6-a448-450df3c26c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:16.710602 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-1a8931b3-cd73-49f6-a448-450df3c26c66 None None] MFA Rules not processed for user `0777c39686cc4071bd5bf9325dd7e571`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:16.715596 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-1a8931b3-cd73-49f6-a448-450df3c26c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:16.816977 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1017/2038] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:16 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:45:16.850735 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7e51b-afde-400b-bbea-3b577c024e7c admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:16.856385 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:16.931359 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7e51b-afde-400b-bbea-3b577c024e7c admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:16.935321 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adaf4d5-0ee7-44c7-af02-54bb36df8980 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:16.935924 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adaf4d5-0ee7-44c7-af02-54bb36df8980 service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:16.936246 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adaf4d5-0ee7-44c7-af02-54bb36df8980 service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:16.936544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adaf4d5-0ee7-44c7-af02-54bb36df8980 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:16.946260 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adaf4d5-0ee7-44c7-af02-54bb36df8980 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:16.948047 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adaf4d5-0ee7-44c7-af02-54bb36df8980 service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:16.963802 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1022/2039] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:45:16 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:45:16.994485 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8931b3-cd73-49f6-a448-450df3c26c66 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:17.056143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8931b3-cd73-49f6-a448-450df3c26c66 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:17.060776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c6d28b-9f88-41aa-8f99-17aa12d647d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:17.061402 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6d28b-9f88-41aa-8f99-17aa12d647d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:17.061683 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6d28b-9f88-41aa-8f99-17aa12d647d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:17.061958 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6d28b-9f88-41aa-8f99-17aa12d647d5 admin admin] PATH_INFO: `/v3/users/62bec362c71848e5b786ba1cde67a776` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:17.063087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c6d28b-9f88-41aa-8f99-17aa12d647d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62bec362c71848e5b786ba1cde67a776)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:17.065054 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c6d28b-9f88-41aa-8f99-17aa12d647d5 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:17.080349 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:17.118817 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-30c6d28b-9f88-41aa-8f99-17aa12d647d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62bec362c71848e5b786ba1cde67a776'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:17.119476 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-30c6d28b-9f88-41aa-8f99-17aa12d647d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62bec362c71848e5b786ba1cde67a776'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:17.120115 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-30c6d28b-9f88-41aa-8f99-17aa12d647d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62bec362c71848e5b786ba1cde67a776'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:17.128819 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-30c6d28b-9f88-41aa-8f99-17aa12d647d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62bec362c71848e5b786ba1cde67a776'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:17.135902 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-30c6d28b-9f88-41aa-8f99-17aa12d647d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62bec362c71848e5b786ba1cde67a776'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:17.153924 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1018/2040] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:45:16 2022] DELETE /identity/v3/users/62bec362c71848e5b786ba1cde67a776 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:17.158301 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9933d-c4c3-4298-a266-8564a8c90b12 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:17.158792 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9933d-c4c3-4298-a266-8564a8c90b12 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:17.158901 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9933d-c4c3-4298-a266-8564a8c90b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:17.178579 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-bae9933d-c4c3-4298-a266-8564a8c90b12 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:17.179794 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-bae9933d-c4c3-4298-a266-8564a8c90b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:17.241489 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1023/2041] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:45:17.253235 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c6d28b-9f88-41aa-8f99-17aa12d647d5 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:17.254741 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-30c6d28b-9f88-41aa-8f99-17aa12d647d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:17.318098 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c6d28b-9f88-41aa-8f99-17aa12d647d5 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:17.321752 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55ba713-ddf8-4946-ba2a-1a4b8971fe4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:17.322358 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ba713-ddf8-4946-ba2a-1a4b8971fe4a service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:17.322637 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ba713-ddf8-4946-ba2a-1a4b8971fe4a service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:17.322911 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ba713-ddf8-4946-ba2a-1a4b8971fe4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:17.332880 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55ba713-ddf8-4946-ba2a-1a4b8971fe4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:17.334313 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55ba713-ddf8-4946-ba2a-1a4b8971fe4a service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:17.345253 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1019/2042] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:45:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:45:17.609401 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae9933d-c4c3-4298-a266-8564a8c90b12 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:17.610938 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-bae9933d-c4c3-4298-a266-8564a8c90b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:17.626835 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae9933d-c4c3-4298-a266-8564a8c90b12 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:17.631628 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5148ad6c-ee93-4b59-90b7-f17ed4cf6719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:17.632438 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148ad6c-ee93-4b59-90b7-f17ed4cf6719 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:17.632844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148ad6c-ee93-4b59-90b7-f17ed4cf6719 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:17.633221 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148ad6c-ee93-4b59-90b7-f17ed4cf6719 admin admin] PATH_INFO: `/v3/projects/fe1305efdd04422dbc1a297de85f4319` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:17.634840 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5148ad6c-ee93-4b59-90b7-f17ed4cf6719 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe1305efdd04422dbc1a297de85f4319)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:17.637446 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5148ad6c-ee93-4b59-90b7-f17ed4cf6719 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:17.692060 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-5148ad6c-ee93-4b59-90b7-f17ed4cf6719 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe1305efdd04422dbc1a297de85f4319', 'request_id': 'req-5148ad6c-ee93-4b59-90b7-f17ed4cf6719'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:17.697552 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-5148ad6c-ee93-4b59-90b7-f17ed4cf6719 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe1305efdd04422dbc1a297de85f4319', 'request_id': 'req-5148ad6c-ee93-4b59-90b7-f17ed4cf6719'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:17.704714 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-5148ad6c-ee93-4b59-90b7-f17ed4cf6719 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe1305efdd04422dbc1a297de85f4319', 'request_id': 'req-5148ad6c-ee93-4b59-90b7-f17ed4cf6719'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:17.717141 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-5148ad6c-ee93-4b59-90b7-f17ed4cf6719 admin admin] The token cache is being invalidate because project fe1305efdd04422dbc1a297de85f4319 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:45:17.717436 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-5148ad6c-ee93-4b59-90b7-f17ed4cf6719 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:17.718108 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-5148ad6c-ee93-4b59-90b7-f17ed4cf6719 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:17.719605 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1024/2043] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:45:17 2022] DELETE /identity/v3/projects/fe1305efdd04422dbc1a297de85f4319 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:17.723600 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55ba713-ddf8-4946-ba2a-1a4b8971fe4a service neutron] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:17.734126 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55ba713-ddf8-4946-ba2a-1a4b8971fe4a service neutron] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:17.791147 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc0edc-6177-4469-9907-3cea932d5c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:17.791890 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc0edc-6177-4469-9907-3cea932d5c48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:17.792220 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc0edc-6177-4469-9907-3cea932d5c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:17.792498 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc0edc-6177-4469-9907-3cea932d5c48 admin admin] PATH_INFO: `/v3/users/0777c39686cc4071bd5bf9325dd7e571` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:17.794230 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fc0edc-6177-4469-9907-3cea932d5c48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0777c39686cc4071bd5bf9325dd7e571)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:17.796759 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fc0edc-6177-4469-9907-3cea932d5c48 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:17.842215 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:17.850419 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-a2fc0edc-6177-4469-9907-3cea932d5c48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0777c39686cc4071bd5bf9325dd7e571'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:17.852339 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-a2fc0edc-6177-4469-9907-3cea932d5c48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0777c39686cc4071bd5bf9325dd7e571'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:17.853033 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-a2fc0edc-6177-4469-9907-3cea932d5c48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0777c39686cc4071bd5bf9325dd7e571'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:17.861720 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-a2fc0edc-6177-4469-9907-3cea932d5c48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0777c39686cc4071bd5bf9325dd7e571'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:17.873508 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-a2fc0edc-6177-4469-9907-3cea932d5c48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0777c39686cc4071bd5bf9325dd7e571'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:17.891547 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1020/2044] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:45:17 2022] DELETE /identity/v3/users/0777c39686cc4071bd5bf9325dd7e571 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:18.175199 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5148ad6c-ee93-4b59-90b7-f17ed4cf6719 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:18.176659 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-5148ad6c-ee93-4b59-90b7-f17ed4cf6719 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:18.189364 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5148ad6c-ee93-4b59-90b7-f17ed4cf6719 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:18.221367 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:18.252971 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94804d72-c12a-4350-b652-4b2da2fdfb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:18.253530 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94804d72-c12a-4350-b652-4b2da2fdfb75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:18.253809 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94804d72-c12a-4350-b652-4b2da2fdfb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:18.254080 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94804d72-c12a-4350-b652-4b2da2fdfb75 admin admin] PATH_INFO: `/v3/projects/95a29dab38fa4b0db6a70390b8a03425` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:18.255201 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94804d72-c12a-4350-b652-4b2da2fdfb75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95a29dab38fa4b0db6a70390b8a03425)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:18.257180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94804d72-c12a-4350-b652-4b2da2fdfb75 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:18.310790 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-94804d72-c12a-4350-b652-4b2da2fdfb75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95a29dab38fa4b0db6a70390b8a03425', 'request_id': 'req-94804d72-c12a-4350-b652-4b2da2fdfb75'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:18.316852 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-94804d72-c12a-4350-b652-4b2da2fdfb75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95a29dab38fa4b0db6a70390b8a03425', 'request_id': 'req-94804d72-c12a-4350-b652-4b2da2fdfb75'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:18.325606 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-94804d72-c12a-4350-b652-4b2da2fdfb75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95a29dab38fa4b0db6a70390b8a03425', 'request_id': 'req-94804d72-c12a-4350-b652-4b2da2fdfb75'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:18.338369 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-94804d72-c12a-4350-b652-4b2da2fdfb75 admin admin] The token cache is being invalidate because project 95a29dab38fa4b0db6a70390b8a03425 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:45:18.338657 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-94804d72-c12a-4350-b652-4b2da2fdfb75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:18.339329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-94804d72-c12a-4350-b652-4b2da2fdfb75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:18.341495 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1025/2045] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:45:18 2022] DELETE /identity/v3/projects/95a29dab38fa4b0db6a70390b8a03425 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:18.347578 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e8c02a-2875-43ec-bbd5-e17c53498d9e None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:18.348095 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e8c02a-2875-43ec-bbd5-e17c53498d9e None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:18.348526 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e8c02a-2875-43ec-bbd5-e17c53498d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:18.370121 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-32e8c02a-2875-43ec-bbd5-e17c53498d9e None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:18.371551 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-32e8c02a-2875-43ec-bbd5-e17c53498d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:18.459679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1021/2046] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:45:18.468273 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7c37d-d909-40ca-ad45-6d94fe621034 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:18.468553 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7c37d-d909-40ca-ad45-6d94fe621034 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:18.468820 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7c37d-d909-40ca-ad45-6d94fe621034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:18.490971 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-17e7c37d-d909-40ca-ad45-6d94fe621034 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:18.492360 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-17e7c37d-d909-40ca-ad45-6d94fe621034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:18.503746 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1026/2047] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:45:18.505920 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e8c02a-2875-43ec-bbd5-e17c53498d9e None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:18.517557 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e8c02a-2875-43ec-bbd5-e17c53498d9e None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:18.523014 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34884426-4a3b-4110-8c1b-1f3ab45e1d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:18.524232 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34884426-4a3b-4110-8c1b-1f3ab45e1d63 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:18.524458 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34884426-4a3b-4110-8c1b-1f3ab45e1d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:18.524597 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34884426-4a3b-4110-8c1b-1f3ab45e1d63 admin admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:18.525292 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34884426-4a3b-4110-8c1b-1f3ab45e1d63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:45:18.525605 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34884426-4a3b-4110-8c1b-1f3ab45e1d63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:18.527249 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34884426-4a3b-4110-8c1b-1f3ab45e1d63 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:18.543080 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1022/2048] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:45:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:45:18.544077 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7c37d-d909-40ca-ad45-6d94fe621034 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:18.553272 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7c37d-d909-40ca-ad45-6d94fe621034 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:18.558743 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da614dfb-79f5-41bd-a0f0-c2ae1e3bc593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:18.559352 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da614dfb-79f5-41bd-a0f0-c2ae1e3bc593 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:18.559632 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da614dfb-79f5-41bd-a0f0-c2ae1e3bc593 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:18.559902 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da614dfb-79f5-41bd-a0f0-c2ae1e3bc593 admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:18.560566 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da614dfb-79f5-41bd-a0f0-c2ae1e3bc593 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:18.562485 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da614dfb-79f5-41bd-a0f0-c2ae1e3bc593 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:18.595723 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1027/2049] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:45:18 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:45:18.599944 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34884426-4a3b-4110-8c1b-1f3ab45e1d63 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:18.610104 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34884426-4a3b-4110-8c1b-1f3ab45e1d63 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:18.675000 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99184793-a8f4-44c8-8e3e-193863777f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:18.675705 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99184793-a8f4-44c8-8e3e-193863777f49 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:18.676093 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99184793-a8f4-44c8-8e3e-193863777f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:18.676444 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99184793-a8f4-44c8-8e3e-193863777f49 admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:18.677149 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99184793-a8f4-44c8-8e3e-193863777f49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:18.679157 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99184793-a8f4-44c8-8e3e-193863777f49 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:18.700708 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1023/2050] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:45:18 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:45:18.704659 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da614dfb-79f5-41bd-a0f0-c2ae1e3bc593 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:18.715100 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da614dfb-79f5-41bd-a0f0-c2ae1e3bc593 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:18.720009 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d555841c-7508-4ee6-8404-6911753c2745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:18.720596 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555841c-7508-4ee6-8404-6911753c2745 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:18.720840 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555841c-7508-4ee6-8404-6911753c2745 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:18.721108 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555841c-7508-4ee6-8404-6911753c2745 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:18.721778 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d555841c-7508-4ee6-8404-6911753c2745 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:18.723492 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d555841c-7508-4ee6-8404-6911753c2745 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:18.734589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1028/2051] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:45:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:45:18.738638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99184793-a8f4-44c8-8e3e-193863777f49 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:18.749532 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99184793-a8f4-44c8-8e3e-193863777f49 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:18.754359 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862fe7d2-5c49-48a4-b391-bb675bbb5372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:18.754886 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862fe7d2-5c49-48a4-b391-bb675bbb5372 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:18.755175 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862fe7d2-5c49-48a4-b391-bb675bbb5372 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:18.755746 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862fe7d2-5c49-48a4-b391-bb675bbb5372 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:18.756256 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862fe7d2-5c49-48a4-b391-bb675bbb5372 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:18.757868 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862fe7d2-5c49-48a4-b391-bb675bbb5372 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:18.772007 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1024/2052] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:45:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:45:18.774207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d555841c-7508-4ee6-8404-6911753c2745 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:18.785742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d555841c-7508-4ee6-8404-6911753c2745 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:18.790719 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6483026f-e167-4905-8484-7775450dd01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:18.791672 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6483026f-e167-4905-8484-7775450dd01a admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:18.791844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6483026f-e167-4905-8484-7775450dd01a admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:18.792143 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6483026f-e167-4905-8484-7775450dd01a admin admin] PATH_INFO: `/v3/projects/cb5c726e4360417b955a292a712ab3cd/users/1f0b81cf3a294218834b4f98400b7abf/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:18.807607 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6483026f-e167-4905-8484-7775450dd01a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb5c726e4360417b955a292a712ab3cd, user_id=1f0b81cf3a294218834b4f98400b7abf, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:18.810881 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6483026f-e167-4905-8484-7775450dd01a admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:18.823842 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1029/2053] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:45:18 2022] PUT /identity/v3/projects/cb5c726e4360417b955a292a712ab3cd/users/1f0b81cf3a294218834b4f98400b7abf/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:18.834902 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427ab07-99d4-46f2-818d-8f391dddd66d None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:18.835183 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427ab07-99d4-46f2-818d-8f391dddd66d None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:18.835462 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427ab07-99d4-46f2-818d-8f391dddd66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:18.867033 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-4427ab07-99d4-46f2-818d-8f391dddd66d None None] MFA Rules not processed for user `1f0b81cf3a294218834b4f98400b7abf`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:18.868415 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-4427ab07-99d4-46f2-818d-8f391dddd66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:18.883861 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:18.954973 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1025/2054] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:18 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:45:18.959433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6483026f-e167-4905-8484-7775450dd01a admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:19.028972 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:19.033504 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6483026f-e167-4905-8484-7775450dd01a admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:19.042022 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6bc5e-f235-4f2a-905e-1547d26019c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:19.043138 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6bc5e-f235-4f2a-905e-1547d26019c6 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:19.043898 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6bc5e-f235-4f2a-905e-1547d26019c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:19.044534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6bc5e-f235-4f2a-905e-1547d26019c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:19.046989 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf6bc5e-f235-4f2a-905e-1547d26019c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:19.049513 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf6bc5e-f235-4f2a-905e-1547d26019c6 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:19.100257 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1030/2055] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:45:18 2022] POST /identity/v3/projects => generated 387 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:45:19.103877 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427ab07-99d4-46f2-818d-8f391dddd66d None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:19.126083 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427ab07-99d4-46f2-818d-8f391dddd66d None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:19.195828 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d51caf-6e66-42a9-bf9f-7bbb7d51a138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:19.196784 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d51caf-6e66-42a9-bf9f-7bbb7d51a138 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:19.196977 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d51caf-6e66-42a9-bf9f-7bbb7d51a138 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:19.197191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d51caf-6e66-42a9-bf9f-7bbb7d51a138 admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:19.197822 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d51caf-6e66-42a9-bf9f-7bbb7d51a138 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:19.200223 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d51caf-6e66-42a9-bf9f-7bbb7d51a138 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:19.232575 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1026/2056] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:45:19 2022] POST /identity/v3/users => generated 300 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:45:19.237329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6bc5e-f235-4f2a-905e-1547d26019c6 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:19.252781 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6bc5e-f235-4f2a-905e-1547d26019c6 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:19.265244 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f134d8a-e34c-4aa9-aa05-23def2acde43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:19.266258 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f134d8a-e34c-4aa9-aa05-23def2acde43 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:19.266987 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f134d8a-e34c-4aa9-aa05-23def2acde43 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:19.267443 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f134d8a-e34c-4aa9-aa05-23def2acde43 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:19.268452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f134d8a-e34c-4aa9-aa05-23def2acde43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:19.271074 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f134d8a-e34c-4aa9-aa05-23def2acde43 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:19.304016 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1031/2057] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 23:45:19.306004 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d51caf-6e66-42a9-bf9f-7bbb7d51a138 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:19.316748 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d51caf-6e66-42a9-bf9f-7bbb7d51a138 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:19.321872 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f366c021-f93e-471e-a386-7c5a29eb3805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:19.322528 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f366c021-f93e-471e-a386-7c5a29eb3805 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:19.322809 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f366c021-f93e-471e-a386-7c5a29eb3805 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:19.323084 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f366c021-f93e-471e-a386-7c5a29eb3805 admin admin] PATH_INFO: `/v3/projects/a6f6dcbd0755423aa4d35da71ab958b9/users/d27f63b55dec40f78ae7004ab8b9fdd3/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:19.338391 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f366c021-f93e-471e-a386-7c5a29eb3805 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6f6dcbd0755423aa4d35da71ab958b9, user_id=d27f63b55dec40f78ae7004ab8b9fdd3, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:19.341679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f366c021-f93e-471e-a386-7c5a29eb3805 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:19.354924 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1027/2058] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:45:19 2022] PUT /identity/v3/projects/a6f6dcbd0755423aa4d35da71ab958b9/users/d27f63b55dec40f78ae7004ab8b9fdd3/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:19.359811 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f134d8a-e34c-4aa9-aa05-23def2acde43 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:19.369727 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f134d8a-e34c-4aa9-aa05-23def2acde43 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:19.426372 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38bd73c-45a1-4b26-8b23-8746d767b81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:19.426972 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38bd73c-45a1-4b26-8b23-8746d767b81c admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:19.427389 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38bd73c-45a1-4b26-8b23-8746d767b81c admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:19.427655 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38bd73c-45a1-4b26-8b23-8746d767b81c admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:19.428338 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38bd73c-45a1-4b26-8b23-8746d767b81c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:19.430005 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38bd73c-45a1-4b26-8b23-8746d767b81c admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:19.440109 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1032/2059] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:45:19.444304 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f366c021-f93e-471e-a386-7c5a29eb3805 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:19.458105 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f366c021-f93e-471e-a386-7c5a29eb3805 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:19.462771 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e470d-0662-4017-bb7d-304f90883b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:19.463424 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e470d-0662-4017-bb7d-304f90883b82 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:19.463707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e470d-0662-4017-bb7d-304f90883b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:19.464026 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e470d-0662-4017-bb7d-304f90883b82 admin admin] PATH_INFO: `/v3/projects/a6f6dcbd0755423aa4d35da71ab958b9/users/d27f63b55dec40f78ae7004ab8b9fdd3/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:19.466773 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e470d-0662-4017-bb7d-304f90883b82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6f6dcbd0755423aa4d35da71ab958b9, user_id=d27f63b55dec40f78ae7004ab8b9fdd3, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:19.469464 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e470d-0662-4017-bb7d-304f90883b82 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:19.483320 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1028/2060] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:45:19 2022] PUT /identity/v3/projects/a6f6dcbd0755423aa4d35da71ab958b9/users/d27f63b55dec40f78ae7004ab8b9fdd3/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:19.492503 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e043de2-e074-4379-a334-f33ac6238aef None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:19.492785 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e043de2-e074-4379-a334-f33ac6238aef None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:19.493052 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e043de2-e074-4379-a334-f33ac6238aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:19.520505 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-5e043de2-e074-4379-a334-f33ac6238aef None None] MFA Rules not processed for user `d27f63b55dec40f78ae7004ab8b9fdd3`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:19.521640 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-5e043de2-e074-4379-a334-f33ac6238aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:19.588243 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1033/2061] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:19 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:45:19.599574 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e470d-0662-4017-bb7d-304f90883b82 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:19.601413 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-432e470d-0662-4017-bb7d-304f90883b82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:19.659558 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e470d-0662-4017-bb7d-304f90883b82 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:19.663652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d07e2dd-01ef-4b3a-9103-939c3a28d3fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:19.664287 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07e2dd-01ef-4b3a-9103-939c3a28d3fa service cinder] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:19.664569 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07e2dd-01ef-4b3a-9103-939c3a28d3fa service cinder] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:19.664841 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07e2dd-01ef-4b3a-9103-939c3a28d3fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:19.673997 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d07e2dd-01ef-4b3a-9103-939c3a28d3fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:19.675447 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d07e2dd-01ef-4b3a-9103-939c3a28d3fa service cinder] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:19.686002 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1029/2062] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:45:19 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:45:20.255001 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e043de2-e074-4379-a334-f33ac6238aef None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:20.311292 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:20.327665 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e043de2-e074-4379-a334-f33ac6238aef None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:20.336315 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc551f4-8549-4682-8687-48530f11c495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:20.336953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc551f4-8549-4682-8687-48530f11c495 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:20.337224 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc551f4-8549-4682-8687-48530f11c495 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:20.337814 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc551f4-8549-4682-8687-48530f11c495 admin admin] PATH_INFO: `/v3/users/1f0b81cf3a294218834b4f98400b7abf` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:20.338643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc551f4-8549-4682-8687-48530f11c495 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f0b81cf3a294218834b4f98400b7abf)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:20.340785 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc551f4-8549-4682-8687-48530f11c495 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:20.387537 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-edc551f4-8549-4682-8687-48530f11c495 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f0b81cf3a294218834b4f98400b7abf'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:20.388229 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-edc551f4-8549-4682-8687-48530f11c495 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f0b81cf3a294218834b4f98400b7abf'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:20.403264 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-edc551f4-8549-4682-8687-48530f11c495 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f0b81cf3a294218834b4f98400b7abf'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:20.418883 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-edc551f4-8549-4682-8687-48530f11c495 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f0b81cf3a294218834b4f98400b7abf'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:20.419796 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-edc551f4-8549-4682-8687-48530f11c495 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f0b81cf3a294218834b4f98400b7abf'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:20.434552 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1034/2063] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:45:20 2022] DELETE /identity/v3/users/1f0b81cf3a294218834b4f98400b7abf => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:20.439132 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6890332-3048-4237-8fd8-41934b5d3411 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:20.439670 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6890332-3048-4237-8fd8-41934b5d3411 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:20.439758 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6890332-3048-4237-8fd8-41934b5d3411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:20.454274 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:20.461552 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-b6890332-3048-4237-8fd8-41934b5d3411 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:20.463521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-b6890332-3048-4237-8fd8-41934b5d3411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:20.527819 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1030/2064] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:45:20.539039 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc551f4-8549-4682-8687-48530f11c495 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:20.540625 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-edc551f4-8549-4682-8687-48530f11c495 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:20.600540 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc551f4-8549-4682-8687-48530f11c495 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:20.604145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90337a84-8531-4712-b3ce-da45169befd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:20.604746 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90337a84-8531-4712-b3ce-da45169befd2 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:20.605547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90337a84-8531-4712-b3ce-da45169befd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:20.605834 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90337a84-8531-4712-b3ce-da45169befd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:20.614700 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90337a84-8531-4712-b3ce-da45169befd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:20.616155 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90337a84-8531-4712-b3ce-da45169befd2 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:20.628255 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1035/2065] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:45:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:45:20.940073 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6890332-3048-4237-8fd8-41934b5d3411 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:20.942039 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-b6890332-3048-4237-8fd8-41934b5d3411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:20.958642 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6890332-3048-4237-8fd8-41934b5d3411 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:20.963538 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeafd5ad-dbb9-4fc1-8c36-0ce36fea1af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:20.964988 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeafd5ad-dbb9-4fc1-8c36-0ce36fea1af3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:20.965562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeafd5ad-dbb9-4fc1-8c36-0ce36fea1af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:20.965784 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeafd5ad-dbb9-4fc1-8c36-0ce36fea1af3 admin admin] PATH_INFO: `/v3/projects/cb5c726e4360417b955a292a712ab3cd` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:20.967141 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeafd5ad-dbb9-4fc1-8c36-0ce36fea1af3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb5c726e4360417b955a292a712ab3cd)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:20.969466 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeafd5ad-dbb9-4fc1-8c36-0ce36fea1af3 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:21.025238 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-eeafd5ad-dbb9-4fc1-8c36-0ce36fea1af3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb5c726e4360417b955a292a712ab3cd', 'request_id': 'req-eeafd5ad-dbb9-4fc1-8c36-0ce36fea1af3'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:21.031501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-eeafd5ad-dbb9-4fc1-8c36-0ce36fea1af3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb5c726e4360417b955a292a712ab3cd', 'request_id': 'req-eeafd5ad-dbb9-4fc1-8c36-0ce36fea1af3'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:21.039303 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-eeafd5ad-dbb9-4fc1-8c36-0ce36fea1af3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb5c726e4360417b955a292a712ab3cd', 'request_id': 'req-eeafd5ad-dbb9-4fc1-8c36-0ce36fea1af3'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:21.051020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-eeafd5ad-dbb9-4fc1-8c36-0ce36fea1af3 admin admin] The token cache is being invalidate because project cb5c726e4360417b955a292a712ab3cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:45:21.051309 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-eeafd5ad-dbb9-4fc1-8c36-0ce36fea1af3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:21.051955 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-eeafd5ad-dbb9-4fc1-8c36-0ce36fea1af3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:21.053471 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1031/2066] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:45:20 2022] DELETE /identity/v3/projects/cb5c726e4360417b955a292a712ab3cd => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:21.057615 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90337a84-8531-4712-b3ce-da45169befd2 service neutron] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:21.068133 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90337a84-8531-4712-b3ce-da45169befd2 service neutron] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:21.126578 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1089df-3bdc-4c59-8919-a6b204330563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:21.127376 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1089df-3bdc-4c59-8919-a6b204330563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:21.127853 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1089df-3bdc-4c59-8919-a6b204330563 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:21.128252 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1089df-3bdc-4c59-8919-a6b204330563 admin admin] PATH_INFO: `/v3/users/d27f63b55dec40f78ae7004ab8b9fdd3` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:21.129629 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1089df-3bdc-4c59-8919-a6b204330563 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d27f63b55dec40f78ae7004ab8b9fdd3)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:21.131800 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1089df-3bdc-4c59-8919-a6b204330563 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:21.148559 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:21.182673 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-bc1089df-3bdc-4c59-8919-a6b204330563 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd27f63b55dec40f78ae7004ab8b9fdd3'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:21.183334 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-bc1089df-3bdc-4c59-8919-a6b204330563 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd27f63b55dec40f78ae7004ab8b9fdd3'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:21.195761 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-bc1089df-3bdc-4c59-8919-a6b204330563 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd27f63b55dec40f78ae7004ab8b9fdd3'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:21.204210 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-bc1089df-3bdc-4c59-8919-a6b204330563 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd27f63b55dec40f78ae7004ab8b9fdd3'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:21.204881 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-bc1089df-3bdc-4c59-8919-a6b204330563 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd27f63b55dec40f78ae7004ab8b9fdd3'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:21.222302 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1036/2067] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:45:21 2022] DELETE /identity/v3/users/d27f63b55dec40f78ae7004ab8b9fdd3 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:21.547277 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeafd5ad-dbb9-4fc1-8c36-0ce36fea1af3 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:21.548886 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-eeafd5ad-dbb9-4fc1-8c36-0ce36fea1af3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:21.564649 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeafd5ad-dbb9-4fc1-8c36-0ce36fea1af3 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:21.625365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ee735-e24d-4530-87ec-15ed56871de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:21.626011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ee735-e24d-4530-87ec-15ed56871de0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:21.626270 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ee735-e24d-4530-87ec-15ed56871de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:21.626543 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ee735-e24d-4530-87ec-15ed56871de0 admin admin] PATH_INFO: `/v3/projects/a6f6dcbd0755423aa4d35da71ab958b9` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:21.627656 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410ee735-e24d-4530-87ec-15ed56871de0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6f6dcbd0755423aa4d35da71ab958b9)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:21.629606 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410ee735-e24d-4530-87ec-15ed56871de0 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:21.663451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:21.676291 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-410ee735-e24d-4530-87ec-15ed56871de0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6f6dcbd0755423aa4d35da71ab958b9', 'request_id': 'req-410ee735-e24d-4530-87ec-15ed56871de0'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:21.681792 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-410ee735-e24d-4530-87ec-15ed56871de0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6f6dcbd0755423aa4d35da71ab958b9', 'request_id': 'req-410ee735-e24d-4530-87ec-15ed56871de0'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:21.689536 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-410ee735-e24d-4530-87ec-15ed56871de0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6f6dcbd0755423aa4d35da71ab958b9', 'request_id': 'req-410ee735-e24d-4530-87ec-15ed56871de0'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:21.700997 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-410ee735-e24d-4530-87ec-15ed56871de0 admin admin] The token cache is being invalidate because project a6f6dcbd0755423aa4d35da71ab958b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:45:21.701286 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-410ee735-e24d-4530-87ec-15ed56871de0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:21.701914 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-410ee735-e24d-4530-87ec-15ed56871de0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:21.703390 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1032/2068] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:45:21 2022] DELETE /identity/v3/projects/a6f6dcbd0755423aa4d35da71ab958b9 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:21.708373 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81243d-849b-46f6-9eb6-969a5fb0e1bd None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:21.708812 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81243d-849b-46f6-9eb6-969a5fb0e1bd None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:21.708925 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81243d-849b-46f6-9eb6-969a5fb0e1bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:21.729296 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-0e81243d-849b-46f6-9eb6-969a5fb0e1bd None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:21.730517 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-0e81243d-849b-46f6-9eb6-969a5fb0e1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:21.788650 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1037/2069] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:45:21.797542 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50da6e-bf4a-416d-885e-f4136f34a061 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:21.797847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50da6e-bf4a-416d-885e-f4136f34a061 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:21.798141 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50da6e-bf4a-416d-885e-f4136f34a061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:21.817792 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-4c50da6e-bf4a-416d-885e-f4136f34a061 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:21.818994 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-4c50da6e-bf4a-416d-885e-f4136f34a061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:21.826948 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1033/2070] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:45:21.831285 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e81243d-849b-46f6-9eb6-969a5fb0e1bd None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:21.840981 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e81243d-849b-46f6-9eb6-969a5fb0e1bd None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:21.845518 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a79bdd-11df-44c6-9546-20eba948f3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:21.846112 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a79bdd-11df-44c6-9546-20eba948f3f9 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:21.846391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a79bdd-11df-44c6-9546-20eba948f3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:21.846663 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a79bdd-11df-44c6-9546-20eba948f3f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:21.847308 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a79bdd-11df-44c6-9546-20eba948f3f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:45:21.847610 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a79bdd-11df-44c6-9546-20eba948f3f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:21.849245 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a79bdd-11df-44c6-9546-20eba948f3f9 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:21.860943 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1038/2071] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:45:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:45:21.864929 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50da6e-bf4a-416d-885e-f4136f34a061 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:21.876855 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50da6e-bf4a-416d-885e-f4136f34a061 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:21.881459 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4852557-7982-488a-9804-1a41c26a233d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:21.882047 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4852557-7982-488a-9804-1a41c26a233d admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:21.882325 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4852557-7982-488a-9804-1a41c26a233d admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:21.882595 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4852557-7982-488a-9804-1a41c26a233d admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:21.883209 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4852557-7982-488a-9804-1a41c26a233d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:21.885157 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4852557-7982-488a-9804-1a41c26a233d admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:21.912671 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1034/2072] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:45:21 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:45:21.916834 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a79bdd-11df-44c6-9546-20eba948f3f9 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:21.927490 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a79bdd-11df-44c6-9546-20eba948f3f9 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:21.983429 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6866b687-41cb-42a5-b916-6c7d9daaff6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:21.984083 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866b687-41cb-42a5-b916-6c7d9daaff6b admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:21.984427 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866b687-41cb-42a5-b916-6c7d9daaff6b admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:21.984705 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866b687-41cb-42a5-b916-6c7d9daaff6b admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:21.985336 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6866b687-41cb-42a5-b916-6c7d9daaff6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:21.987263 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6866b687-41cb-42a5-b916-6c7d9daaff6b admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:22.011835 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1039/2073] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:45:21 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:45:22.013320 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4852557-7982-488a-9804-1a41c26a233d admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:22.023861 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4852557-7982-488a-9804-1a41c26a233d admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:22.028941 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fca7f3-2164-4547-a89d-4ca565e0049c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:22.029726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fca7f3-2164-4547-a89d-4ca565e0049c admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:22.030007 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fca7f3-2164-4547-a89d-4ca565e0049c admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:22.030304 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fca7f3-2164-4547-a89d-4ca565e0049c admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:22.030957 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fca7f3-2164-4547-a89d-4ca565e0049c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:22.032636 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fca7f3-2164-4547-a89d-4ca565e0049c admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:22.044141 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1035/2074] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:45:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:45:22.048632 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6866b687-41cb-42a5-b916-6c7d9daaff6b admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:22.062351 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6866b687-41cb-42a5-b916-6c7d9daaff6b admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:22.067092 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d28564b-3974-4634-ba50-d4ba3c3dceda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:22.067698 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d28564b-3974-4634-ba50-d4ba3c3dceda admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:22.068393 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d28564b-3974-4634-ba50-d4ba3c3dceda admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:22.068393 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d28564b-3974-4634-ba50-d4ba3c3dceda admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:22.068912 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d28564b-3974-4634-ba50-d4ba3c3dceda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:22.070528 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d28564b-3974-4634-ba50-d4ba3c3dceda admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:22.073187 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:22.084596 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1040/2075] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:45:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:45:22.088862 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fca7f3-2164-4547-a89d-4ca565e0049c admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:22.098789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fca7f3-2164-4547-a89d-4ca565e0049c admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:22.103463 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b121e5fd-c55c-41b7-b5b4-0db759c61d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:22.104199 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b121e5fd-c55c-41b7-b5b4-0db759c61d1b admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:22.104481 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b121e5fd-c55c-41b7-b5b4-0db759c61d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:22.104755 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b121e5fd-c55c-41b7-b5b4-0db759c61d1b admin admin] PATH_INFO: `/v3/projects/731f642b2f914908a1e2af1c80462f8f/users/3c942b1fcb884310b67c097e6cbfb3d0/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:22.120111 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b121e5fd-c55c-41b7-b5b4-0db759c61d1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=731f642b2f914908a1e2af1c80462f8f, user_id=3c942b1fcb884310b67c097e6cbfb3d0, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:22.123375 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b121e5fd-c55c-41b7-b5b4-0db759c61d1b admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:22.136135 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1036/2076] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:45:22 2022] PUT /identity/v3/projects/731f642b2f914908a1e2af1c80462f8f/users/3c942b1fcb884310b67c097e6cbfb3d0/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:45:22.147383 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa57b7b-72a5-4111-a8e1-af1fd6d19ea9 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:22.147662 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa57b7b-72a5-4111-a8e1-af1fd6d19ea9 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:22.147927 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa57b7b-72a5-4111-a8e1-af1fd6d19ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:22.175799 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-afa57b7b-72a5-4111-a8e1-af1fd6d19ea9 None None] MFA Rules not processed for user `3c942b1fcb884310b67c097e6cbfb3d0`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:22.176977 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-afa57b7b-72a5-4111-a8e1-af1fd6d19ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:22.242233 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1041/2077] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:45:22.252963 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b121e5fd-c55c-41b7-b5b4-0db759c61d1b admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:22.254617 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-b121e5fd-c55c-41b7-b5b4-0db759c61d1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:22.331254 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b121e5fd-c55c-41b7-b5b4-0db759c61d1b admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:22.334937 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c8199-366b-401a-abd6-3880b32ac2f8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c08c497a14c426f9cfe0dd8c8100a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:22.335541 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c8199-366b-401a-abd6-3880b32ac2f8 service glance] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:22.335817 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c8199-366b-401a-abd6-3880b32ac2f8 service glance] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:22.336149 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c8199-366b-401a-abd6-3880b32ac2f8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:22.345150 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c8199-366b-401a-abd6-3880b32ac2f8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:22.346572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c8199-366b-401a-abd6-3880b32ac2f8 service glance] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:22.356403 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1037/2078] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:45:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 23:45:26.743737 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa57b7b-72a5-4111-a8e1-af1fd6d19ea9 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:26.745294 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-afa57b7b-72a5-4111-a8e1-af1fd6d19ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:26.836348 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa57b7b-72a5-4111-a8e1-af1fd6d19ea9 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:26.843720 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac66644-62c6-49ea-ab0d-a9eceb8019d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:26.844391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac66644-62c6-49ea-ab0d-a9eceb8019d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:26.844668 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac66644-62c6-49ea-ab0d-a9eceb8019d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:26.844936 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac66644-62c6-49ea-ab0d-a9eceb8019d0 admin admin] PATH_INFO: `/v3/users/56bbec5df7354cc1bed911a1e4575c29` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:26.846054 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac66644-62c6-49ea-ab0d-a9eceb8019d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56bbec5df7354cc1bed911a1e4575c29)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:26.848264 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac66644-62c6-49ea-ab0d-a9eceb8019d0 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:26.894778 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-cac66644-62c6-49ea-ab0d-a9eceb8019d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56bbec5df7354cc1bed911a1e4575c29'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:26.895428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-cac66644-62c6-49ea-ab0d-a9eceb8019d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56bbec5df7354cc1bed911a1e4575c29'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:26.907562 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-cac66644-62c6-49ea-ab0d-a9eceb8019d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56bbec5df7354cc1bed911a1e4575c29'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:26.915310 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-cac66644-62c6-49ea-ab0d-a9eceb8019d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56bbec5df7354cc1bed911a1e4575c29'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:26.915923 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-cac66644-62c6-49ea-ab0d-a9eceb8019d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56bbec5df7354cc1bed911a1e4575c29'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:26.928739 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1042/2079] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:45:26 2022] DELETE /identity/v3/users/56bbec5df7354cc1bed911a1e4575c29 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:26.933607 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bdfd6e-2756-4636-b1f0-b79dd89708f4 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:26.934122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bdfd6e-2756-4636-b1f0-b79dd89708f4 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:26.934223 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bdfd6e-2756-4636-b1f0-b79dd89708f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:26.954162 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-34bdfd6e-2756-4636-b1f0-b79dd89708f4 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:26.955589 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-34bdfd6e-2756-4636-b1f0-b79dd89708f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:26.992869 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:27.024990 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1038/2080] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:45:27.036582 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac66644-62c6-49ea-ab0d-a9eceb8019d0 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:27.038097 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-cac66644-62c6-49ea-ab0d-a9eceb8019d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:27.049428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:27.116809 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac66644-62c6-49ea-ab0d-a9eceb8019d0 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:27.120526 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098d024a-6af7-4e98-aaeb-380bd9740698 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:27.121133 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d024a-6af7-4e98-aaeb-380bd9740698 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:27.121437 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d024a-6af7-4e98-aaeb-380bd9740698 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:27.121708 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d024a-6af7-4e98-aaeb-380bd9740698 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:27.131569 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098d024a-6af7-4e98-aaeb-380bd9740698 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:27.133060 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098d024a-6af7-4e98-aaeb-380bd9740698 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:27.143046 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1043/2081] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:45:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:45:27.540572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bdfd6e-2756-4636-b1f0-b79dd89708f4 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:27.542217 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-34bdfd6e-2756-4636-b1f0-b79dd89708f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:27.577405 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bdfd6e-2756-4636-b1f0-b79dd89708f4 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:27.582408 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae3f18f-30ea-4e85-bcce-d46165da1879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:27.583058 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae3f18f-30ea-4e85-bcce-d46165da1879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:27.583360 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae3f18f-30ea-4e85-bcce-d46165da1879 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:27.583617 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae3f18f-30ea-4e85-bcce-d46165da1879 admin admin] PATH_INFO: `/v3/projects/b9c2cad2bb2149698dabf1c88508a9b0` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:27.584824 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae3f18f-30ea-4e85-bcce-d46165da1879 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9c2cad2bb2149698dabf1c88508a9b0)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:27.586852 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae3f18f-30ea-4e85-bcce-d46165da1879 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:27.640267 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-2ae3f18f-30ea-4e85-bcce-d46165da1879 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9c2cad2bb2149698dabf1c88508a9b0', 'request_id': 'req-2ae3f18f-30ea-4e85-bcce-d46165da1879'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:27.646590 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-2ae3f18f-30ea-4e85-bcce-d46165da1879 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9c2cad2bb2149698dabf1c88508a9b0', 'request_id': 'req-2ae3f18f-30ea-4e85-bcce-d46165da1879'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:27.660104 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-2ae3f18f-30ea-4e85-bcce-d46165da1879 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9c2cad2bb2149698dabf1c88508a9b0', 'request_id': 'req-2ae3f18f-30ea-4e85-bcce-d46165da1879'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:27.687853 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-2ae3f18f-30ea-4e85-bcce-d46165da1879 admin admin] The token cache is being invalidate because project b9c2cad2bb2149698dabf1c88508a9b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:45:27.688878 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-2ae3f18f-30ea-4e85-bcce-d46165da1879 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:27.689727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-2ae3f18f-30ea-4e85-bcce-d46165da1879 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:27.694062 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1039/2082] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:45:27 2022] DELETE /identity/v3/projects/b9c2cad2bb2149698dabf1c88508a9b0 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:27.696247 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098d024a-6af7-4e98-aaeb-380bd9740698 service neutron] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:27.708226 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098d024a-6af7-4e98-aaeb-380bd9740698 service neutron] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:27.781555 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdbf856-8ac1-4f16-bf30-d2e851d175f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:27.781555 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdbf856-8ac1-4f16-bf30-d2e851d175f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:27.781555 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdbf856-8ac1-4f16-bf30-d2e851d175f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:27.781555 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdbf856-8ac1-4f16-bf30-d2e851d175f1 admin admin] PATH_INFO: `/v3/users/d8658f4423144dec9cfa89c0ae92d866` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:27.782628 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdbf856-8ac1-4f16-bf30-d2e851d175f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8658f4423144dec9cfa89c0ae92d866)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:27.785067 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdbf856-8ac1-4f16-bf30-d2e851d175f1 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:27.825188 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae3f18f-30ea-4e85-bcce-d46165da1879 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:27.828387 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-2ae3f18f-30ea-4e85-bcce-d46165da1879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:27.841323 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae3f18f-30ea-4e85-bcce-d46165da1879 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:27.846859 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5345783-2cf6-4928-9013-571eb52ed08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:27.847645 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5345783-2cf6-4928-9013-571eb52ed08c admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:27.848088 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5345783-2cf6-4928-9013-571eb52ed08c admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:27.848450 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5345783-2cf6-4928-9013-571eb52ed08c admin admin] PATH_INFO: `/v3/users/9f1cab1669f74ce4a5e7bda7acddf175` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:27.849697 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5345783-2cf6-4928-9013-571eb52ed08c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f1cab1669f74ce4a5e7bda7acddf175)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:27.851750 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5345783-2cf6-4928-9013-571eb52ed08c admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:27.874025 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-5fdbf856-8ac1-4f16-bf30-d2e851d175f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8658f4423144dec9cfa89c0ae92d866'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:27.874884 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-5fdbf856-8ac1-4f16-bf30-d2e851d175f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8658f4423144dec9cfa89c0ae92d866'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:27.895424 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-5fdbf856-8ac1-4f16-bf30-d2e851d175f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8658f4423144dec9cfa89c0ae92d866'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:27.914669 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-5fdbf856-8ac1-4f16-bf30-d2e851d175f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8658f4423144dec9cfa89c0ae92d866'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:27.917127 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-5fdbf856-8ac1-4f16-bf30-d2e851d175f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8658f4423144dec9cfa89c0ae92d866'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:27.943932 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-b5345783-2cf6-4928-9013-571eb52ed08c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f1cab1669f74ce4a5e7bda7acddf175'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:27.945594 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1044/2083] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:45:27 2022] DELETE /identity/v3/users/d8658f4423144dec9cfa89c0ae92d866 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:27.945890 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-b5345783-2cf6-4928-9013-571eb52ed08c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f1cab1669f74ce4a5e7bda7acddf175'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:27.946857 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-b5345783-2cf6-4928-9013-571eb52ed08c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f1cab1669f74ce4a5e7bda7acddf175'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:27.956299 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-b5345783-2cf6-4928-9013-571eb52ed08c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f1cab1669f74ce4a5e7bda7acddf175'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:27.967181 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-b5345783-2cf6-4928-9013-571eb52ed08c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f1cab1669f74ce4a5e7bda7acddf175'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:27.986821 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1040/2084] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:45:27 2022] DELETE /identity/v3/users/9f1cab1669f74ce4a5e7bda7acddf175 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:27.991201 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04615b52-3f7e-49ca-94a4-657d1f1a3a3d None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:27.991494 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04615b52-3f7e-49ca-94a4-657d1f1a3a3d None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:27.991774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04615b52-3f7e-49ca-94a4-657d1f1a3a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:28.011637 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-04615b52-3f7e-49ca-94a4-657d1f1a3a3d None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:28.012926 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-04615b52-3f7e-49ca-94a4-657d1f1a3a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:28.039240 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:28.096086 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1045/2085] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:45:28.107168 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5345783-2cf6-4928-9013-571eb52ed08c admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:28.121472 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-b5345783-2cf6-4928-9013-571eb52ed08c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:28.166170 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:28.186648 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5345783-2cf6-4928-9013-571eb52ed08c admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:28.190278 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55535cf-0bac-43d8-bace-8cbc98a628c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:28.190882 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55535cf-0bac-43d8-bace-8cbc98a628c1 service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:28.191163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55535cf-0bac-43d8-bace-8cbc98a628c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:28.191441 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55535cf-0bac-43d8-bace-8cbc98a628c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:28.201065 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55535cf-0bac-43d8-bace-8cbc98a628c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:28.202640 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55535cf-0bac-43d8-bace-8cbc98a628c1 service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:28.213148 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1041/2086] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:45:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 23:45:28.458971 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04615b52-3f7e-49ca-94a4-657d1f1a3a3d None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:28.460656 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-04615b52-3f7e-49ca-94a4-657d1f1a3a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:28.475919 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04615b52-3f7e-49ca-94a4-657d1f1a3a3d None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:28.481373 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ce9918-3023-48fe-a757-7738bb10800e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:28.482019 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce9918-3023-48fe-a757-7738bb10800e admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:28.482303 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce9918-3023-48fe-a757-7738bb10800e admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:28.483292 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce9918-3023-48fe-a757-7738bb10800e admin admin] PATH_INFO: `/v3/projects/472844468f254c488a96c2707cf34bbf` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:28.484054 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ce9918-3023-48fe-a757-7738bb10800e admin admin] RBAC: Authorizing `identity:delete_project(project_id=472844468f254c488a96c2707cf34bbf)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:28.486042 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ce9918-3023-48fe-a757-7738bb10800e admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:28.565604 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-c4ce9918-3023-48fe-a757-7738bb10800e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '472844468f254c488a96c2707cf34bbf', 'request_id': 'req-c4ce9918-3023-48fe-a757-7738bb10800e'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:28.571828 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-c4ce9918-3023-48fe-a757-7738bb10800e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '472844468f254c488a96c2707cf34bbf', 'request_id': 'req-c4ce9918-3023-48fe-a757-7738bb10800e'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:28.580906 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-c4ce9918-3023-48fe-a757-7738bb10800e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '472844468f254c488a96c2707cf34bbf', 'request_id': 'req-c4ce9918-3023-48fe-a757-7738bb10800e'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:28.592421 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-c4ce9918-3023-48fe-a757-7738bb10800e admin admin] The token cache is being invalidate because project 472844468f254c488a96c2707cf34bbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:45:28.592776 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-c4ce9918-3023-48fe-a757-7738bb10800e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:28.593484 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-c4ce9918-3023-48fe-a757-7738bb10800e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:28.595324 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1046/2087] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:45:28 2022] DELETE /identity/v3/projects/472844468f254c488a96c2707cf34bbf => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:28.600386 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f7043-22fe-41b2-8d28-9af574334cce None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:28.600812 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f7043-22fe-41b2-8d28-9af574334cce None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:28.601193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f7043-22fe-41b2-8d28-9af574334cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:28.622085 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-453f7043-22fe-41b2-8d28-9af574334cce None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:28.623876 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-453f7043-22fe-41b2-8d28-9af574334cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:28.658534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ce9918-3023-48fe-a757-7738bb10800e admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:28.659985 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-c4ce9918-3023-48fe-a757-7738bb10800e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:28.710456 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1042/2088] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:45:28.711902 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e88be-32eb-48d9-ab37-a97032f7766e None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:28.712308 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e88be-32eb-48d9-ab37-a97032f7766e None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:28.712496 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e88be-32eb-48d9-ab37-a97032f7766e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:28.732496 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-ed6e88be-32eb-48d9-ab37-a97032f7766e None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:28.733723 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-ed6e88be-32eb-48d9-ab37-a97032f7766e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:28.741635 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1043/2089] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:45:28.743791 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ce9918-3023-48fe-a757-7738bb10800e admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:28.745900 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e88be-32eb-48d9-ab37-a97032f7766e None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:28.750985 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20792ba2-dc3e-4e3a-9ca8-5624f81307b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:28.751624 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20792ba2-dc3e-4e3a-9ca8-5624f81307b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:28.751905 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20792ba2-dc3e-4e3a-9ca8-5624f81307b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:28.752202 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20792ba2-dc3e-4e3a-9ca8-5624f81307b5 admin admin] PATH_INFO: `/v3/projects/ca5162376893424183ab35bad8fddac3` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:28.753347 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20792ba2-dc3e-4e3a-9ca8-5624f81307b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca5162376893424183ab35bad8fddac3)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:28.755260 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20792ba2-dc3e-4e3a-9ca8-5624f81307b5 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:28.755899 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e88be-32eb-48d9-ab37-a97032f7766e None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:28.760546 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3e8603-ef13-40d1-8eda-03bbeaba1770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:28.761145 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e8603-ef13-40d1-8eda-03bbeaba1770 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:28.761426 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e8603-ef13-40d1-8eda-03bbeaba1770 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:28.761705 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e8603-ef13-40d1-8eda-03bbeaba1770 admin admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:28.762393 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3e8603-ef13-40d1-8eda-03bbeaba1770 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:45:28.762726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3e8603-ef13-40d1-8eda-03bbeaba1770 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:28.764362 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3e8603-ef13-40d1-8eda-03bbeaba1770 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:28.779599 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1044/2090] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:45:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:45:28.780696 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3e8603-ef13-40d1-8eda-03bbeaba1770 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:28.791981 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3e8603-ef13-40d1-8eda-03bbeaba1770 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:28.807253 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-20792ba2-dc3e-4e3a-9ca8-5624f81307b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca5162376893424183ab35bad8fddac3', 'request_id': 'req-20792ba2-dc3e-4e3a-9ca8-5624f81307b5'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:28.811740 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:28.814113 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-20792ba2-dc3e-4e3a-9ca8-5624f81307b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca5162376893424183ab35bad8fddac3', 'request_id': 'req-20792ba2-dc3e-4e3a-9ca8-5624f81307b5'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:28.823666 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-20792ba2-dc3e-4e3a-9ca8-5624f81307b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca5162376893424183ab35bad8fddac3', 'request_id': 'req-20792ba2-dc3e-4e3a-9ca8-5624f81307b5'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:28.836745 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-20792ba2-dc3e-4e3a-9ca8-5624f81307b5 admin admin] The token cache is being invalidate because project ca5162376893424183ab35bad8fddac3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:45:28.837086 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-20792ba2-dc3e-4e3a-9ca8-5624f81307b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:28.837720 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-20792ba2-dc3e-4e3a-9ca8-5624f81307b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:28.839394 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1047/2091] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:45:28 2022] DELETE /identity/v3/projects/ca5162376893424183ab35bad8fddac3 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:45:28.864465 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06faf062-6cde-48b5-a1f0-efc51f17ba61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:28.865213 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06faf062-6cde-48b5-a1f0-efc51f17ba61 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:28.865571 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06faf062-6cde-48b5-a1f0-efc51f17ba61 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:28.865916 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06faf062-6cde-48b5-a1f0-efc51f17ba61 admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:28.866639 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06faf062-6cde-48b5-a1f0-efc51f17ba61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:28.868741 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06faf062-6cde-48b5-a1f0-efc51f17ba61 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:28.915885 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1045/2092] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:45:28 2022] POST /identity/v3/projects => generated 379 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:45:28.921172 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20792ba2-dc3e-4e3a-9ca8-5624f81307b5 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:28.925056 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-20792ba2-dc3e-4e3a-9ca8-5624f81307b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:28.939234 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20792ba2-dc3e-4e3a-9ca8-5624f81307b5 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:29.018171 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5babd8f-e9bf-4e68-bba3-0413967f4e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:29.018910 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5babd8f-e9bf-4e68-bba3-0413967f4e67 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:29.019434 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5babd8f-e9bf-4e68-bba3-0413967f4e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:29.019805 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5babd8f-e9bf-4e68-bba3-0413967f4e67 admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:29.020752 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5babd8f-e9bf-4e68-bba3-0413967f4e67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:29.023776 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5babd8f-e9bf-4e68-bba3-0413967f4e67 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:29.062786 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1048/2093] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:45:28 2022] POST /identity/v3/users => generated 290 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:45:29.064292 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06faf062-6cde-48b5-a1f0-efc51f17ba61 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:29.075563 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06faf062-6cde-48b5-a1f0-efc51f17ba61 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:29.080530 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c435ee8a-1f69-497c-8c06-b1ec1db6a49f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:29.081190 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435ee8a-1f69-497c-8c06-b1ec1db6a49f admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:29.081471 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435ee8a-1f69-497c-8c06-b1ec1db6a49f admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:29.081740 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435ee8a-1f69-497c-8c06-b1ec1db6a49f admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:29.082392 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c435ee8a-1f69-497c-8c06-b1ec1db6a49f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:29.084168 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c435ee8a-1f69-497c-8c06-b1ec1db6a49f admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:29.099900 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1046/2094] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 23:45:29.101259 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5babd8f-e9bf-4e68-bba3-0413967f4e67 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:29.113142 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5babd8f-e9bf-4e68-bba3-0413967f4e67 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:29.118270 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69bce6-181c-44aa-8f8a-f642774d17f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:29.119082 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69bce6-181c-44aa-8f8a-f642774d17f0 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:29.119359 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69bce6-181c-44aa-8f8a-f642774d17f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:29.119626 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69bce6-181c-44aa-8f8a-f642774d17f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:29.121615 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69bce6-181c-44aa-8f8a-f642774d17f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:29.122202 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69bce6-181c-44aa-8f8a-f642774d17f0 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:29.135997 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1049/2095] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:45:29.137895 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c435ee8a-1f69-497c-8c06-b1ec1db6a49f admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:29.149510 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c435ee8a-1f69-497c-8c06-b1ec1db6a49f admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:29.155798 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e061ed10-2ee2-4c0a-9f17-2f821fcdeaa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:29.156618 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e061ed10-2ee2-4c0a-9f17-2f821fcdeaa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:29.156987 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e061ed10-2ee2-4c0a-9f17-2f821fcdeaa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:29.157331 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e061ed10-2ee2-4c0a-9f17-2f821fcdeaa0 admin admin] PATH_INFO: `/v3/projects/c4a5af5a1c894a15b4461520170b4cef/users/9471221301784a7c82f5592b17948cac/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:29.184775 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e061ed10-2ee2-4c0a-9f17-2f821fcdeaa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4a5af5a1c894a15b4461520170b4cef, user_id=9471221301784a7c82f5592b17948cac, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:29.187763 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e061ed10-2ee2-4c0a-9f17-2f821fcdeaa0 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:29.201579 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1047/2096] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:45:29 2022] PUT /identity/v3/projects/c4a5af5a1c894a15b4461520170b4cef/users/9471221301784a7c82f5592b17948cac/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:45:29.211772 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ef34c-b9c8-4953-9556-3f354fb3b330 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:29.212099 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ef34c-b9c8-4953-9556-3f354fb3b330 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:29.212377 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ef34c-b9c8-4953-9556-3f354fb3b330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:29.242254 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-df8ef34c-b9c8-4953-9556-3f354fb3b330 None None] MFA Rules not processed for user `9471221301784a7c82f5592b17948cac`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:29.243675 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-df8ef34c-b9c8-4953-9556-3f354fb3b330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:29.322419 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1050/2097] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:45:29.327395 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e061ed10-2ee2-4c0a-9f17-2f821fcdeaa0 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:29.337089 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:29.389655 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e061ed10-2ee2-4c0a-9f17-2f821fcdeaa0 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:29.394368 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b7df95-1d51-4a8e-9232-6d8a8f7d1ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:29.394981 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7df95-1d51-4a8e-9232-6d8a8f7d1ff5 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:29.395255 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7df95-1d51-4a8e-9232-6d8a8f7d1ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:29.395525 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7df95-1d51-4a8e-9232-6d8a8f7d1ff5 admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:29.396179 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b7df95-1d51-4a8e-9232-6d8a8f7d1ff5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:29.398078 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b7df95-1d51-4a8e-9232-6d8a8f7d1ff5 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:29.424835 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1048/2098] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:45:29 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:45:29.429073 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ef34c-b9c8-4953-9556-3f354fb3b330 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:29.439647 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ef34c-b9c8-4953-9556-3f354fb3b330 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:29.489367 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:29.500658 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58da27-06c2-4a89-a334-642271d36fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:29.501261 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58da27-06c2-4a89-a334-642271d36fb3 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:29.501536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58da27-06c2-4a89-a334-642271d36fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:29.501796 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58da27-06c2-4a89-a334-642271d36fb3 admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:29.502424 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae58da27-06c2-4a89-a334-642271d36fb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:29.504380 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae58da27-06c2-4a89-a334-642271d36fb3 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:29.525192 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1051/2099] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:45:29 2022] POST /identity/v3/users => generated 296 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:45:29.530879 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b7df95-1d51-4a8e-9232-6d8a8f7d1ff5 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:29.541794 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b7df95-1d51-4a8e-9232-6d8a8f7d1ff5 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:29.546405 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b4783-7bd9-4a4a-99d7-2c5b2f83d87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:29.546999 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b4783-7bd9-4a4a-99d7-2c5b2f83d87b admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:29.547274 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b4783-7bd9-4a4a-99d7-2c5b2f83d87b admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:29.547540 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b4783-7bd9-4a4a-99d7-2c5b2f83d87b admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:29.548209 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8b4783-7bd9-4a4a-99d7-2c5b2f83d87b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:29.549830 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8b4783-7bd9-4a4a-99d7-2c5b2f83d87b admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:29.564095 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1049/2100] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:45:29.565493 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58da27-06c2-4a89-a334-642271d36fb3 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:29.576231 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58da27-06c2-4a89-a334-642271d36fb3 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:29.580720 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a72708d-229e-4d61-b020-0f118cfbb8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:29.581364 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a72708d-229e-4d61-b020-0f118cfbb8fb admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:29.581642 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a72708d-229e-4d61-b020-0f118cfbb8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:29.581913 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a72708d-229e-4d61-b020-0f118cfbb8fb admin admin] PATH_INFO: `/v3/projects/6c47f7903cb24aca917f0191f995638c/users/1a24c844ceac4dcb824a1c5f7cab583e/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:29.596662 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a72708d-229e-4d61-b020-0f118cfbb8fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c47f7903cb24aca917f0191f995638c, user_id=1a24c844ceac4dcb824a1c5f7cab583e, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:29.599315 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a72708d-229e-4d61-b020-0f118cfbb8fb admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:29.614497 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1052/2101] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:45:29 2022] PUT /identity/v3/projects/6c47f7903cb24aca917f0191f995638c/users/1a24c844ceac4dcb824a1c5f7cab583e/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:45:29.616698 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b4783-7bd9-4a4a-99d7-2c5b2f83d87b admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:29.626651 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b4783-7bd9-4a4a-99d7-2c5b2f83d87b admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:29.686388 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fc945c-ea0c-4fe1-a3a4-7c6c76725364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:29.686996 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc945c-ea0c-4fe1-a3a4-7c6c76725364 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:29.687278 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc945c-ea0c-4fe1-a3a4-7c6c76725364 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:29.687553 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc945c-ea0c-4fe1-a3a4-7c6c76725364 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:29.688246 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fc945c-ea0c-4fe1-a3a4-7c6c76725364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:29.690242 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fc945c-ea0c-4fe1-a3a4-7c6c76725364 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:29.703062 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1050/2102] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:45:29.707525 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a72708d-229e-4d61-b020-0f118cfbb8fb admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:29.718160 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a72708d-229e-4d61-b020-0f118cfbb8fb admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:29.722861 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f863559-acca-4541-9b21-7ea4c89e420a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:29.723539 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f863559-acca-4541-9b21-7ea4c89e420a admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:29.723814 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f863559-acca-4541-9b21-7ea4c89e420a admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:29.724116 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f863559-acca-4541-9b21-7ea4c89e420a admin admin] PATH_INFO: `/v3/projects/6c47f7903cb24aca917f0191f995638c/users/1a24c844ceac4dcb824a1c5f7cab583e/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:29.726176 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f863559-acca-4541-9b21-7ea4c89e420a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c47f7903cb24aca917f0191f995638c, user_id=1a24c844ceac4dcb824a1c5f7cab583e, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:29.729129 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f863559-acca-4541-9b21-7ea4c89e420a admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:29.739937 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1053/2103] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:45:29 2022] PUT /identity/v3/projects/6c47f7903cb24aca917f0191f995638c/users/1a24c844ceac4dcb824a1c5f7cab583e/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:29.752691 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4eb5b-16ed-4668-8983-0ee721afe0d3 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:29.753017 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4eb5b-16ed-4668-8983-0ee721afe0d3 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:29.753289 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4eb5b-16ed-4668-8983-0ee721afe0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:29.781080 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-10f4eb5b-16ed-4668-8983-0ee721afe0d3 None None] MFA Rules not processed for user `1a24c844ceac4dcb824a1c5f7cab583e`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:29.782291 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-10f4eb5b-16ed-4668-8983-0ee721afe0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:29.874567 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1051/2104] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:29 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:45:29.886566 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f863559-acca-4541-9b21-7ea4c89e420a admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:29.889008 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-7f863559-acca-4541-9b21-7ea4c89e420a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:29.952920 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f863559-acca-4541-9b21-7ea4c89e420a admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:29.956558 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45219994-1e18-4a39-8c02-af2846fffabf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:29.957163 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45219994-1e18-4a39-8c02-af2846fffabf service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:29.957443 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45219994-1e18-4a39-8c02-af2846fffabf service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:29.957716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45219994-1e18-4a39-8c02-af2846fffabf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:29.967613 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45219994-1e18-4a39-8c02-af2846fffabf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:29.969125 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45219994-1e18-4a39-8c02-af2846fffabf service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:29.981022 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1054/2105] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:45:29 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:45:30.364573 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f4eb5b-16ed-4668-8983-0ee721afe0d3 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:30.370096 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:30.445033 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f4eb5b-16ed-4668-8983-0ee721afe0d3 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:30.450368 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9518c3-aa13-48b3-a051-fbfe649e2713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:30.451017 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9518c3-aa13-48b3-a051-fbfe649e2713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:30.451353 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9518c3-aa13-48b3-a051-fbfe649e2713 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:30.451630 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9518c3-aa13-48b3-a051-fbfe649e2713 admin admin] PATH_INFO: `/v3/users/9471221301784a7c82f5592b17948cac` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:30.452830 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9518c3-aa13-48b3-a051-fbfe649e2713 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9471221301784a7c82f5592b17948cac)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:30.454778 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9518c3-aa13-48b3-a051-fbfe649e2713 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:30.504718 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-ec9518c3-aa13-48b3-a051-fbfe649e2713 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9471221301784a7c82f5592b17948cac'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:30.505387 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-ec9518c3-aa13-48b3-a051-fbfe649e2713 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9471221301784a7c82f5592b17948cac'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:30.505988 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-ec9518c3-aa13-48b3-a051-fbfe649e2713 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9471221301784a7c82f5592b17948cac'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:30.513409 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-ec9518c3-aa13-48b3-a051-fbfe649e2713 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9471221301784a7c82f5592b17948cac'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:30.520711 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-ec9518c3-aa13-48b3-a051-fbfe649e2713 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9471221301784a7c82f5592b17948cac'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:30.538688 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1052/2106] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:45:30 2022] DELETE /identity/v3/users/9471221301784a7c82f5592b17948cac => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:30.543836 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3876002-ac81-4675-8f19-eac3788609ff None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:30.544179 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3876002-ac81-4675-8f19-eac3788609ff None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:30.544487 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3876002-ac81-4675-8f19-eac3788609ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:30.563674 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-c3876002-ac81-4675-8f19-eac3788609ff None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:30.564975 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-c3876002-ac81-4675-8f19-eac3788609ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:30.624980 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1055/2107] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:45:30.635450 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9518c3-aa13-48b3-a051-fbfe649e2713 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:30.636927 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-ec9518c3-aa13-48b3-a051-fbfe649e2713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:30.696275 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9518c3-aa13-48b3-a051-fbfe649e2713 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:30.699855 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd0050-92ac-48c1-88cd-317c0192b98c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:30.700485 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd0050-92ac-48c1-88cd-317c0192b98c service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:30.700764 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd0050-92ac-48c1-88cd-317c0192b98c service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:30.701035 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd0050-92ac-48c1-88cd-317c0192b98c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:30.709867 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd0050-92ac-48c1-88cd-317c0192b98c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:30.711287 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd0050-92ac-48c1-88cd-317c0192b98c service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:30.721186 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1053/2108] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:45:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:45:30.996682 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3876002-ac81-4675-8f19-eac3788609ff None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:30.998230 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-c3876002-ac81-4675-8f19-eac3788609ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:31.010337 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3876002-ac81-4675-8f19-eac3788609ff None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:31.016166 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37afd702-aef7-44f8-b631-9ae2f7a18965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:31.017132 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37afd702-aef7-44f8-b631-9ae2f7a18965 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:31.017624 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37afd702-aef7-44f8-b631-9ae2f7a18965 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:31.018104 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37afd702-aef7-44f8-b631-9ae2f7a18965 admin admin] PATH_INFO: `/v3/projects/c4a5af5a1c894a15b4461520170b4cef` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:31.019583 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37afd702-aef7-44f8-b631-9ae2f7a18965 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4a5af5a1c894a15b4461520170b4cef)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:31.021771 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37afd702-aef7-44f8-b631-9ae2f7a18965 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:31.065156 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:31.072909 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-37afd702-aef7-44f8-b631-9ae2f7a18965 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4a5af5a1c894a15b4461520170b4cef', 'request_id': 'req-37afd702-aef7-44f8-b631-9ae2f7a18965'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:31.083622 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-37afd702-aef7-44f8-b631-9ae2f7a18965 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4a5af5a1c894a15b4461520170b4cef', 'request_id': 'req-37afd702-aef7-44f8-b631-9ae2f7a18965'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:31.094319 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-37afd702-aef7-44f8-b631-9ae2f7a18965 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4a5af5a1c894a15b4461520170b4cef', 'request_id': 'req-37afd702-aef7-44f8-b631-9ae2f7a18965'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:31.106941 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-37afd702-aef7-44f8-b631-9ae2f7a18965 admin admin] The token cache is being invalidate because project c4a5af5a1c894a15b4461520170b4cef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:45:31.106941 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-37afd702-aef7-44f8-b631-9ae2f7a18965 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:31.110131 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-37afd702-aef7-44f8-b631-9ae2f7a18965 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:31.112033 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1056/2109] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:45:30 2022] DELETE /identity/v3/projects/c4a5af5a1c894a15b4461520170b4cef => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:31.113129 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd0050-92ac-48c1-88cd-317c0192b98c service neutron] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:31.125324 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd0050-92ac-48c1-88cd-317c0192b98c service neutron] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:31.160645 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:31.201759 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1743ff-335b-4d5f-ba63-7736d5d56412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:31.202402 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1743ff-335b-4d5f-ba63-7736d5d56412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:31.202679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1743ff-335b-4d5f-ba63-7736d5d56412 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:31.202951 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1743ff-335b-4d5f-ba63-7736d5d56412 admin admin] PATH_INFO: `/v3/users/1a24c844ceac4dcb824a1c5f7cab583e` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:31.204131 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1743ff-335b-4d5f-ba63-7736d5d56412 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a24c844ceac4dcb824a1c5f7cab583e)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:31.206075 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1743ff-335b-4d5f-ba63-7736d5d56412 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:31.265884 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-8b1743ff-335b-4d5f-ba63-7736d5d56412 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a24c844ceac4dcb824a1c5f7cab583e'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:31.266595 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-8b1743ff-335b-4d5f-ba63-7736d5d56412 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a24c844ceac4dcb824a1c5f7cab583e'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:31.267455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-8b1743ff-335b-4d5f-ba63-7736d5d56412 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a24c844ceac4dcb824a1c5f7cab583e'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:31.283366 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-8b1743ff-335b-4d5f-ba63-7736d5d56412 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a24c844ceac4dcb824a1c5f7cab583e'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:31.302932 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-8b1743ff-335b-4d5f-ba63-7736d5d56412 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a24c844ceac4dcb824a1c5f7cab583e'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:31.325643 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1054/2110] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:45:31 2022] DELETE /identity/v3/users/1a24c844ceac4dcb824a1c5f7cab583e => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:31.895409 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37afd702-aef7-44f8-b631-9ae2f7a18965 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:31.897058 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-37afd702-aef7-44f8-b631-9ae2f7a18965 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:31.911986 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37afd702-aef7-44f8-b631-9ae2f7a18965 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:31.988637 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fef84e-7855-4170-a347-4f3896fc58bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:31.989427 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fef84e-7855-4170-a347-4f3896fc58bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:31.989707 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fef84e-7855-4170-a347-4f3896fc58bf admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:31.990311 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fef84e-7855-4170-a347-4f3896fc58bf admin admin] PATH_INFO: `/v3/projects/6c47f7903cb24aca917f0191f995638c` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:31.992149 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fef84e-7855-4170-a347-4f3896fc58bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c47f7903cb24aca917f0191f995638c)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:31.994151 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fef84e-7855-4170-a347-4f3896fc58bf admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:32.046312 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-68fef84e-7855-4170-a347-4f3896fc58bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c47f7903cb24aca917f0191f995638c', 'request_id': 'req-68fef84e-7855-4170-a347-4f3896fc58bf'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:32.051748 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-68fef84e-7855-4170-a347-4f3896fc58bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c47f7903cb24aca917f0191f995638c', 'request_id': 'req-68fef84e-7855-4170-a347-4f3896fc58bf'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:32.059952 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-68fef84e-7855-4170-a347-4f3896fc58bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c47f7903cb24aca917f0191f995638c', 'request_id': 'req-68fef84e-7855-4170-a347-4f3896fc58bf'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:32.073220 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-68fef84e-7855-4170-a347-4f3896fc58bf admin admin] The token cache is being invalidate because project 6c47f7903cb24aca917f0191f995638c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:45:32.073506 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-68fef84e-7855-4170-a347-4f3896fc58bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:32.074154 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-68fef84e-7855-4170-a347-4f3896fc58bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:32.076297 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1057/2111] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:45:31 2022] DELETE /identity/v3/projects/6c47f7903cb24aca917f0191f995638c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:32.081134 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc28e1d-6ac1-4fca-8bda-4cf35913ea85 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:32.081582 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc28e1d-6ac1-4fca-8bda-4cf35913ea85 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:32.081700 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc28e1d-6ac1-4fca-8bda-4cf35913ea85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:32.102384 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-7cc28e1d-6ac1-4fca-8bda-4cf35913ea85 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:32.107786 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-7cc28e1d-6ac1-4fca-8bda-4cf35913ea85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:32.167463 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1055/2112] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:45:32.175222 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185aab58-6c13-4c16-bf39-4541ee943373 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:32.175419 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185aab58-6c13-4c16-bf39-4541ee943373 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:32.175688 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185aab58-6c13-4c16-bf39-4541ee943373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:32.201913 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-185aab58-6c13-4c16-bf39-4541ee943373 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:32.203132 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-185aab58-6c13-4c16-bf39-4541ee943373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:32.218730 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1058/2113] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:45:32.223045 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc28e1d-6ac1-4fca-8bda-4cf35913ea85 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:32.233034 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc28e1d-6ac1-4fca-8bda-4cf35913ea85 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:32.237606 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38fa5ee-8a6c-469e-adf1-0cf70d2f715a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:32.238204 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38fa5ee-8a6c-469e-adf1-0cf70d2f715a admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:32.238480 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38fa5ee-8a6c-469e-adf1-0cf70d2f715a admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:32.238754 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38fa5ee-8a6c-469e-adf1-0cf70d2f715a admin admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:32.239405 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38fa5ee-8a6c-469e-adf1-0cf70d2f715a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:45:32.239728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38fa5ee-8a6c-469e-adf1-0cf70d2f715a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:32.241348 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38fa5ee-8a6c-469e-adf1-0cf70d2f715a admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:32.258095 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1056/2114] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:45:32.259319 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185aab58-6c13-4c16-bf39-4541ee943373 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:32.269451 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185aab58-6c13-4c16-bf39-4541ee943373 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:32.274174 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d5951-84b9-4250-bc9f-7136bdbee563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:32.274767 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d5951-84b9-4250-bc9f-7136bdbee563 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:32.275046 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d5951-84b9-4250-bc9f-7136bdbee563 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:32.275316 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d5951-84b9-4250-bc9f-7136bdbee563 admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:32.275937 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24d5951-84b9-4250-bc9f-7136bdbee563 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:32.277962 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24d5951-84b9-4250-bc9f-7136bdbee563 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:32.317883 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1059/2115] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:45:32 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:45:32.319040 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38fa5ee-8a6c-469e-adf1-0cf70d2f715a admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:32.329811 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38fa5ee-8a6c-469e-adf1-0cf70d2f715a admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:32.366807 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:32.401858 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02b766-5d22-46de-90aa-d7118888ea93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:32.402647 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02b766-5d22-46de-90aa-d7118888ea93 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:32.402819 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02b766-5d22-46de-90aa-d7118888ea93 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:32.403078 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02b766-5d22-46de-90aa-d7118888ea93 admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:32.403712 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa02b766-5d22-46de-90aa-d7118888ea93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:32.405668 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa02b766-5d22-46de-90aa-d7118888ea93 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:32.437025 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1057/2116] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:45:32 2022] POST /identity/v3/users => generated 286 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:45:32.438338 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d5951-84b9-4250-bc9f-7136bdbee563 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:32.449510 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d5951-84b9-4250-bc9f-7136bdbee563 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:32.454358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa2ce52-6ee2-47c8-b7d8-b76db1352b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:32.454971 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2ce52-6ee2-47c8-b7d8-b76db1352b61 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:32.455251 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2ce52-6ee2-47c8-b7d8-b76db1352b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:32.455524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2ce52-6ee2-47c8-b7d8-b76db1352b61 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:32.456216 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa2ce52-6ee2-47c8-b7d8-b76db1352b61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:32.457839 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa2ce52-6ee2-47c8-b7d8-b76db1352b61 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:32.473891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1060/2117] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:45:32.475039 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02b766-5d22-46de-90aa-d7118888ea93 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:32.487291 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02b766-5d22-46de-90aa-d7118888ea93 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:32.492612 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b40ab8-91c3-4107-b7a1-b094ec8e71cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:32.493244 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b40ab8-91c3-4107-b7a1-b094ec8e71cf admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:32.493527 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b40ab8-91c3-4107-b7a1-b094ec8e71cf admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:32.493800 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b40ab8-91c3-4107-b7a1-b094ec8e71cf admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:32.494453 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b40ab8-91c3-4107-b7a1-b094ec8e71cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:32.496127 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b40ab8-91c3-4107-b7a1-b094ec8e71cf admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:32.512646 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1058/2118] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:45:32.514180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa2ce52-6ee2-47c8-b7d8-b76db1352b61 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:32.524409 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa2ce52-6ee2-47c8-b7d8-b76db1352b61 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:32.529149 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f204b6-4912-4a8c-ab45-8210e815214e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:32.529856 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f204b6-4912-4a8c-ab45-8210e815214e admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:32.530199 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f204b6-4912-4a8c-ab45-8210e815214e admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:32.530536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f204b6-4912-4a8c-ab45-8210e815214e admin admin] PATH_INFO: `/v3/projects/0b27e2dcce0b4dc4921a48dbf316825d/users/ffbf8a01a6034e7284c2f490121e0ec4/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:32.546799 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f204b6-4912-4a8c-ab45-8210e815214e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b27e2dcce0b4dc4921a48dbf316825d, user_id=ffbf8a01a6034e7284c2f490121e0ec4, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:32.549561 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f204b6-4912-4a8c-ab45-8210e815214e admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:32.553987 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:32.565502 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1061/2119] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:45:32 2022] PUT /identity/v3/projects/0b27e2dcce0b4dc4921a48dbf316825d/users/ffbf8a01a6034e7284c2f490121e0ec4/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:32.577819 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c4bd8-bdea-47bc-9158-6547393b8e02 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:32.577819 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c4bd8-bdea-47bc-9158-6547393b8e02 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:32.577819 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c4bd8-bdea-47bc-9158-6547393b8e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:32.608915 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-9b3c4bd8-bdea-47bc-9158-6547393b8e02 None None] MFA Rules not processed for user `ffbf8a01a6034e7284c2f490121e0ec4`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:32.610298 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-9b3c4bd8-bdea-47bc-9158-6547393b8e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:32.706984 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1059/2120] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:45:32.720516 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f204b6-4912-4a8c-ab45-8210e815214e admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:32.722456 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-17f204b6-4912-4a8c-ab45-8210e815214e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:32.799026 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f204b6-4912-4a8c-ab45-8210e815214e admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:32.806354 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd144f-3ecf-466b-8840-dc77d14a00ac service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c08c497a14c426f9cfe0dd8c8100a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:32.807615 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd144f-3ecf-466b-8840-dc77d14a00ac service glance] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:32.807903 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd144f-3ecf-466b-8840-dc77d14a00ac service glance] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:32.808202 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd144f-3ecf-466b-8840-dc77d14a00ac service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:32.817551 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cd144f-3ecf-466b-8840-dc77d14a00ac service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:32.819004 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cd144f-3ecf-466b-8840-dc77d14a00ac service glance] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:32.829292 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1062/2121] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:45:32 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:45:35.304750 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c4bd8-bdea-47bc-9158-6547393b8e02 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:35.310569 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-9b3c4bd8-bdea-47bc-9158-6547393b8e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:35.379047 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c4bd8-bdea-47bc-9158-6547393b8e02 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:35.385334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43db0bf-bc17-40bc-9ee1-a9796b32bf82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:35.385473 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43db0bf-bc17-40bc-9ee1-a9796b32bf82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:35.385935 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43db0bf-bc17-40bc-9ee1-a9796b32bf82 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:35.386545 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43db0bf-bc17-40bc-9ee1-a9796b32bf82 admin admin] PATH_INFO: `/v3/users/7c852a7d720347be87d8205716dfc15f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:35.388528 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43db0bf-bc17-40bc-9ee1-a9796b32bf82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c852a7d720347be87d8205716dfc15f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:35.391326 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43db0bf-bc17-40bc-9ee1-a9796b32bf82 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:35.462646 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:35.465979 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-c43db0bf-bc17-40bc-9ee1-a9796b32bf82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c852a7d720347be87d8205716dfc15f'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:35.467033 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-c43db0bf-bc17-40bc-9ee1-a9796b32bf82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c852a7d720347be87d8205716dfc15f'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:35.467802 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-c43db0bf-bc17-40bc-9ee1-a9796b32bf82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c852a7d720347be87d8205716dfc15f'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:35.476756 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-c43db0bf-bc17-40bc-9ee1-a9796b32bf82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c852a7d720347be87d8205716dfc15f'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:35.487976 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-c43db0bf-bc17-40bc-9ee1-a9796b32bf82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c852a7d720347be87d8205716dfc15f'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:35.524698 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1060/2122] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:45:35 2022] DELETE /identity/v3/users/7c852a7d720347be87d8205716dfc15f => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:35.749064 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd144f-3ecf-466b-8840-dc77d14a00ac service glance] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:35.762633 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd144f-3ecf-466b-8840-dc77d14a00ac service glance] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:35.824881 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45998242-220f-471c-baa3-c766e49d5bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:35.825870 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45998242-220f-471c-baa3-c766e49d5bcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:35.826358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45998242-220f-471c-baa3-c766e49d5bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:35.826826 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45998242-220f-471c-baa3-c766e49d5bcc admin admin] PATH_INFO: `/v3/projects/b7ebf323aa7c4874bd7d0e2f2873b591` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:35.828347 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45998242-220f-471c-baa3-c766e49d5bcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7ebf323aa7c4874bd7d0e2f2873b591)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:35.830323 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45998242-220f-471c-baa3-c766e49d5bcc admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:35.883607 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-45998242-220f-471c-baa3-c766e49d5bcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7ebf323aa7c4874bd7d0e2f2873b591', 'request_id': 'req-45998242-220f-471c-baa3-c766e49d5bcc'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:35.893552 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-45998242-220f-471c-baa3-c766e49d5bcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7ebf323aa7c4874bd7d0e2f2873b591', 'request_id': 'req-45998242-220f-471c-baa3-c766e49d5bcc'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:35.902914 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-45998242-220f-471c-baa3-c766e49d5bcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7ebf323aa7c4874bd7d0e2f2873b591', 'request_id': 'req-45998242-220f-471c-baa3-c766e49d5bcc'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:35.915146 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-45998242-220f-471c-baa3-c766e49d5bcc admin admin] The token cache is being invalidate because project b7ebf323aa7c4874bd7d0e2f2873b591 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:45:35.915436 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-45998242-220f-471c-baa3-c766e49d5bcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:35.916120 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-45998242-220f-471c-baa3-c766e49d5bcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:35.917616 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1063/2123] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:45:35 2022] DELETE /identity/v3/projects/b7ebf323aa7c4874bd7d0e2f2873b591 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:35.923776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf71bd-5ea6-4e4b-b2f1-7de02e5a7cd7 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:35.924109 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf71bd-5ea6-4e4b-b2f1-7de02e5a7cd7 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:35.924443 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf71bd-5ea6-4e4b-b2f1-7de02e5a7cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:35.944206 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-0abf71bd-5ea6-4e4b-b2f1-7de02e5a7cd7 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:35.945399 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-0abf71bd-5ea6-4e4b-b2f1-7de02e5a7cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:36.006966 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1061/2124] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:45:36.015586 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0e34a-4cd6-4520-88b0-17d22b644a42 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:36.015875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0e34a-4cd6-4520-88b0-17d22b644a42 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:36.016184 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0e34a-4cd6-4520-88b0-17d22b644a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:36.036379 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-1db0e34a-4cd6-4520-88b0-17d22b644a42 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:36.037584 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-1db0e34a-4cd6-4520-88b0-17d22b644a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:36.045515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1064/2125] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:45:36.049778 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf71bd-5ea6-4e4b-b2f1-7de02e5a7cd7 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:36.059771 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf71bd-5ea6-4e4b-b2f1-7de02e5a7cd7 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:36.064467 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4256678a-3c6a-4327-b761-b73dc97805f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:36.065073 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256678a-3c6a-4327-b761-b73dc97805f2 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:36.065355 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256678a-3c6a-4327-b761-b73dc97805f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:36.065663 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256678a-3c6a-4327-b761-b73dc97805f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:36.066401 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4256678a-3c6a-4327-b761-b73dc97805f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:45:36.066718 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4256678a-3c6a-4327-b761-b73dc97805f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:36.068414 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4256678a-3c6a-4327-b761-b73dc97805f2 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:36.081653 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1062/2126] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:45:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:45:36.086163 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0e34a-4cd6-4520-88b0-17d22b644a42 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:36.096021 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0e34a-4cd6-4520-88b0-17d22b644a42 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:36.100673 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a30a20-69db-4563-804f-00d09cafa228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:36.101264 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a30a20-69db-4563-804f-00d09cafa228 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:36.101544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a30a20-69db-4563-804f-00d09cafa228 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:36.101816 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a30a20-69db-4563-804f-00d09cafa228 admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:36.102487 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a30a20-69db-4563-804f-00d09cafa228 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:36.104463 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a30a20-69db-4563-804f-00d09cafa228 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:36.125209 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:36.137250 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1065/2127] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:45:36 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:45:36.137624 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4256678a-3c6a-4327-b761-b73dc97805f2 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:36.148050 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4256678a-3c6a-4327-b761-b73dc97805f2 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:36.207465 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48277143-ae30-456d-a110-cd1f8b104615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:36.208138 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48277143-ae30-456d-a110-cd1f8b104615 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:36.208419 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48277143-ae30-456d-a110-cd1f8b104615 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:36.208692 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48277143-ae30-456d-a110-cd1f8b104615 admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:36.209322 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48277143-ae30-456d-a110-cd1f8b104615 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:36.211339 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48277143-ae30-456d-a110-cd1f8b104615 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:36.233618 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1063/2128] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:45:36 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:45:36.237794 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a30a20-69db-4563-804f-00d09cafa228 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:36.249603 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a30a20-69db-4563-804f-00d09cafa228 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:36.254604 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f373718-b7fa-49c3-ad71-bdf2571a5885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:36.255239 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f373718-b7fa-49c3-ad71-bdf2571a5885 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:36.255547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f373718-b7fa-49c3-ad71-bdf2571a5885 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:36.255817 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f373718-b7fa-49c3-ad71-bdf2571a5885 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:36.256501 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f373718-b7fa-49c3-ad71-bdf2571a5885 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:36.258121 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f373718-b7fa-49c3-ad71-bdf2571a5885 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:36.268914 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1066/2129] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:45:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:45:36.272866 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48277143-ae30-456d-a110-cd1f8b104615 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:36.283025 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48277143-ae30-456d-a110-cd1f8b104615 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:36.287923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba480213-133c-46af-b171-f5c2d1a9bf88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:36.288544 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba480213-133c-46af-b171-f5c2d1a9bf88 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:36.288826 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba480213-133c-46af-b171-f5c2d1a9bf88 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:36.289099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba480213-133c-46af-b171-f5c2d1a9bf88 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:36.289746 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba480213-133c-46af-b171-f5c2d1a9bf88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:36.291340 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba480213-133c-46af-b171-f5c2d1a9bf88 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:36.302460 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1064/2130] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:45:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:45:36.306443 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f373718-b7fa-49c3-ad71-bdf2571a5885 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:36.317142 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f373718-b7fa-49c3-ad71-bdf2571a5885 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:36.321669 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a47c9-0d4f-4624-9499-2396806bb673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:36.322301 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a47c9-0d4f-4624-9499-2396806bb673 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:36.322579 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a47c9-0d4f-4624-9499-2396806bb673 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:36.322848 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a47c9-0d4f-4624-9499-2396806bb673 admin admin] PATH_INFO: `/v3/projects/ec8246e3b9904e7a8fe4a04269ab0afe/users/047293a64a9f4857be1621e77094d367/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:36.338791 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4a47c9-0d4f-4624-9499-2396806bb673 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec8246e3b9904e7a8fe4a04269ab0afe, user_id=047293a64a9f4857be1621e77094d367, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:36.341632 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4a47c9-0d4f-4624-9499-2396806bb673 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:36.355409 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1067/2131] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:45:36 2022] PUT /identity/v3/projects/ec8246e3b9904e7a8fe4a04269ab0afe/users/047293a64a9f4857be1621e77094d367/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:36.361011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b02be1-66fb-4771-a736-6349610208f7 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:36.361578 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b02be1-66fb-4771-a736-6349610208f7 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:36.361578 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b02be1-66fb-4771-a736-6349610208f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:36.381483 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-d2b02be1-66fb-4771-a736-6349610208f7 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:36.382668 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-d2b02be1-66fb-4771-a736-6349610208f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:36.403296 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:36.455798 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1065/2132] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:45:36.460403 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a47c9-0d4f-4624-9499-2396806bb673 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:36.462167 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-1a4a47c9-0d4f-4624-9499-2396806bb673 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:36.520664 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a47c9-0d4f-4624-9499-2396806bb673 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:36.524950 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2543d9-380a-4771-98a3-ef102371dc9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:36.525821 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2543d9-380a-4771-98a3-ef102371dc9d service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:36.526103 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2543d9-380a-4771-98a3-ef102371dc9d service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:36.526372 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2543d9-380a-4771-98a3-ef102371dc9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:36.536888 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2543d9-380a-4771-98a3-ef102371dc9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:36.538335 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2543d9-380a-4771-98a3-ef102371dc9d service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:36.548283 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1068/2133] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:45:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:45:41.223537 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755df643-62b9-4e27-9112-5cec4a6990ee None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:41.224099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755df643-62b9-4e27-9112-5cec4a6990ee None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:41.224184 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755df643-62b9-4e27-9112-5cec4a6990ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:41.256137 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-755df643-62b9-4e27-9112-5cec4a6990ee None None] MFA Rules not processed for user `047293a64a9f4857be1621e77094d367`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:41.257354 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-755df643-62b9-4e27-9112-5cec4a6990ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:41.348683 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1066/2134] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:45:41.361567 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2543d9-380a-4771-98a3-ef102371dc9d service neutron] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:41.363163 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-7f2543d9-380a-4771-98a3-ef102371dc9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:41.482181 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2543d9-380a-4771-98a3-ef102371dc9d service neutron] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:41.489666 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b21c0-eb0a-40a0-ac8e-e5e78dbdc958 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:41.490358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b21c0-eb0a-40a0-ac8e-e5e78dbdc958 service nova] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:41.490709 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b21c0-eb0a-40a0-ac8e-e5e78dbdc958 service nova] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:41.491066 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b21c0-eb0a-40a0-ac8e-e5e78dbdc958 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:41.500787 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b21c0-eb0a-40a0-ac8e-e5e78dbdc958 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:41.502579 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b21c0-eb0a-40a0-ac8e-e5e78dbdc958 service nova] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:41.513000 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1069/2135] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:45:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:45:42.874074 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88506e-5ef3-4e42-b594-7cca90d6a01c None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:42.874629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88506e-5ef3-4e42-b594-7cca90d6a01c None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:42.874769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88506e-5ef3-4e42-b594-7cca90d6a01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:42.896109 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-cc88506e-5ef3-4e42-b594-7cca90d6a01c None None] MFA Rules not processed for user `047293a64a9f4857be1621e77094d367`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:42.897611 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-cc88506e-5ef3-4e42-b594-7cca90d6a01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:42.924344 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1067/2136] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:45:46.006711 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b21c0-eb0a-40a0-ac8e-e5e78dbdc958 service nova] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:46.008265 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-528b21c0-eb0a-40a0-ac8e-e5e78dbdc958 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:46.025567 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b21c0-eb0a-40a0-ac8e-e5e78dbdc958 service nova] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:46.031980 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75099d89-7cd5-4560-b57c-ef54b466fa2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:46.032624 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75099d89-7cd5-4560-b57c-ef54b466fa2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:46.032859 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75099d89-7cd5-4560-b57c-ef54b466fa2f admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:46.033173 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75099d89-7cd5-4560-b57c-ef54b466fa2f admin admin] PATH_INFO: `/v3/users/592d6d184a9b421ea2f2d20232ddea14` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:46.034507 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75099d89-7cd5-4560-b57c-ef54b466fa2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=592d6d184a9b421ea2f2d20232ddea14)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:46.036545 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75099d89-7cd5-4560-b57c-ef54b466fa2f admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:46.086261 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-75099d89-7cd5-4560-b57c-ef54b466fa2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '592d6d184a9b421ea2f2d20232ddea14'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:46.086922 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-75099d89-7cd5-4560-b57c-ef54b466fa2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '592d6d184a9b421ea2f2d20232ddea14'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:46.088223 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:46.115352 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-75099d89-7cd5-4560-b57c-ef54b466fa2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '592d6d184a9b421ea2f2d20232ddea14'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:46.123663 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-75099d89-7cd5-4560-b57c-ef54b466fa2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '592d6d184a9b421ea2f2d20232ddea14'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:46.124514 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-75099d89-7cd5-4560-b57c-ef54b466fa2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '592d6d184a9b421ea2f2d20232ddea14'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:46.138398 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1070/2137] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:45:46 2022] DELETE /identity/v3/users/592d6d184a9b421ea2f2d20232ddea14 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:46.143026 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ab466-3957-4ea5-a482-28e7611141fa None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:46.143521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ab466-3957-4ea5-a482-28e7611141fa None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:46.143756 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ab466-3957-4ea5-a482-28e7611141fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:46.164402 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-d89ab466-3957-4ea5-a482-28e7611141fa None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:46.166067 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-d89ab466-3957-4ea5-a482-28e7611141fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:46.235461 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1068/2138] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:45:46.245184 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75099d89-7cd5-4560-b57c-ef54b466fa2f admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:46.247179 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-75099d89-7cd5-4560-b57c-ef54b466fa2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:46.307690 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75099d89-7cd5-4560-b57c-ef54b466fa2f admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:46.311981 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad572892-b98b-490d-9e9e-2bc259de637a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:46.312609 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad572892-b98b-490d-9e9e-2bc259de637a service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:46.312890 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad572892-b98b-490d-9e9e-2bc259de637a service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:46.313164 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad572892-b98b-490d-9e9e-2bc259de637a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:46.323850 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad572892-b98b-490d-9e9e-2bc259de637a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:46.325959 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad572892-b98b-490d-9e9e-2bc259de637a service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:46.336353 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1071/2139] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:45:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:45:46.690489 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ab466-3957-4ea5-a482-28e7611141fa None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:46.693976 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-d89ab466-3957-4ea5-a482-28e7611141fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:46.711245 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ab466-3957-4ea5-a482-28e7611141fa None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:46.716517 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a214297-89ca-4821-aa6f-c75f226dd897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:46.717489 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a214297-89ca-4821-aa6f-c75f226dd897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:46.717774 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a214297-89ca-4821-aa6f-c75f226dd897 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:46.718050 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a214297-89ca-4821-aa6f-c75f226dd897 admin admin] PATH_INFO: `/v3/projects/fc6705fbfa7342c781f12fede37322b0` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:46.719304 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a214297-89ca-4821-aa6f-c75f226dd897 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc6705fbfa7342c781f12fede37322b0)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:46.721501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a214297-89ca-4821-aa6f-c75f226dd897 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:46.782910 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:46.787276 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-8a214297-89ca-4821-aa6f-c75f226dd897 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc6705fbfa7342c781f12fede37322b0', 'request_id': 'req-8a214297-89ca-4821-aa6f-c75f226dd897'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:46.797220 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-8a214297-89ca-4821-aa6f-c75f226dd897 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc6705fbfa7342c781f12fede37322b0', 'request_id': 'req-8a214297-89ca-4821-aa6f-c75f226dd897'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:46.808173 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-8a214297-89ca-4821-aa6f-c75f226dd897 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc6705fbfa7342c781f12fede37322b0', 'request_id': 'req-8a214297-89ca-4821-aa6f-c75f226dd897'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:46.825151 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-8a214297-89ca-4821-aa6f-c75f226dd897 admin admin] The token cache is being invalidate because project fc6705fbfa7342c781f12fede37322b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:45:46.825532 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-8a214297-89ca-4821-aa6f-c75f226dd897 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:46.826294 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-8a214297-89ca-4821-aa6f-c75f226dd897 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:45:46.828787 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1069/2140] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:45:46 2022] DELETE /identity/v3/projects/fc6705fbfa7342c781f12fede37322b0 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:46.833652 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed2fdd-1649-47e9-9e7b-cabbfac53f1f None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:46.833939 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed2fdd-1649-47e9-9e7b-cabbfac53f1f None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:46.834217 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed2fdd-1649-47e9-9e7b-cabbfac53f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:46.854798 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-c1ed2fdd-1649-47e9-9e7b-cabbfac53f1f None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:46.856979 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-c1ed2fdd-1649-47e9-9e7b-cabbfac53f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:46.941743 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1072/2141] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:45:46.950761 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59aa0a-12e0-4942-b32c-75c75ddae4b8 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:46.951122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59aa0a-12e0-4942-b32c-75c75ddae4b8 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:46.951401 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59aa0a-12e0-4942-b32c-75c75ddae4b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:46.974708 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-3c59aa0a-12e0-4942-b32c-75c75ddae4b8 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:46.976065 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-3c59aa0a-12e0-4942-b32c-75c75ddae4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:46.988835 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1070/2142] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:45:46.993248 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed2fdd-1649-47e9-9e7b-cabbfac53f1f None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:47.004715 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed2fdd-1649-47e9-9e7b-cabbfac53f1f None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:47.009367 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9f64d-63c8-4743-88a4-eaf5ad78f6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:47.009968 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9f64d-63c8-4743-88a4-eaf5ad78f6cf admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:47.010246 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9f64d-63c8-4743-88a4-eaf5ad78f6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:47.010628 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9f64d-63c8-4743-88a4-eaf5ad78f6cf admin admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:47.011323 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9f64d-63c8-4743-88a4-eaf5ad78f6cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:45:47.011719 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9f64d-63c8-4743-88a4-eaf5ad78f6cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:47.013516 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9f64d-63c8-4743-88a4-eaf5ad78f6cf admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:47.032213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1073/2143] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:45:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:45:47.033454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c59aa0a-12e0-4942-b32c-75c75ddae4b8 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:47.044068 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c59aa0a-12e0-4942-b32c-75c75ddae4b8 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:47.048732 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d983dc7e-de6a-4108-bb60-66b007851bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:47.049430 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983dc7e-de6a-4108-bb60-66b007851bf4 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:47.049716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983dc7e-de6a-4108-bb60-66b007851bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:47.049985 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983dc7e-de6a-4108-bb60-66b007851bf4 admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:47.050614 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d983dc7e-de6a-4108-bb60-66b007851bf4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:47.052558 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d983dc7e-de6a-4108-bb60-66b007851bf4 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:47.079397 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1071/2144] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:45:47 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:45:47.083752 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9f64d-63c8-4743-88a4-eaf5ad78f6cf admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:47.094149 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9f64d-63c8-4743-88a4-eaf5ad78f6cf admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:47.147483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:47.158672 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a9d97-8a15-45e5-a58b-b308d61498db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:47.159261 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a9d97-8a15-45e5-a58b-b308d61498db admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:47.159722 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a9d97-8a15-45e5-a58b-b308d61498db admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:47.160196 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a9d97-8a15-45e5-a58b-b308d61498db admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:47.161226 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8a9d97-8a15-45e5-a58b-b308d61498db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:47.163551 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8a9d97-8a15-45e5-a58b-b308d61498db admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:47.191254 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1074/2145] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:45:47 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:45:47.192623 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d983dc7e-de6a-4108-bb60-66b007851bf4 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:47.203460 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d983dc7e-de6a-4108-bb60-66b007851bf4 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:47.208144 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2091c6a4-36dc-430c-9288-230d62d653c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:47.208795 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091c6a4-36dc-430c-9288-230d62d653c3 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:47.209065 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091c6a4-36dc-430c-9288-230d62d653c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:47.209338 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091c6a4-36dc-430c-9288-230d62d653c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:47.210123 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2091c6a4-36dc-430c-9288-230d62d653c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:47.211727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2091c6a4-36dc-430c-9288-230d62d653c3 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:47.223082 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1072/2146] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:45:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:45:47.226865 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a9d97-8a15-45e5-a58b-b308d61498db admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:47.241276 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a9d97-8a15-45e5-a58b-b308d61498db admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:47.246283 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f4f321-300e-4e7b-b948-c980eac611e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:47.246885 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4f321-300e-4e7b-b948-c980eac611e4 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:47.247163 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4f321-300e-4e7b-b948-c980eac611e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:47.247577 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4f321-300e-4e7b-b948-c980eac611e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:47.248257 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f4f321-300e-4e7b-b948-c980eac611e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:47.250757 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f4f321-300e-4e7b-b948-c980eac611e4 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:47.265123 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1075/2147] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:45:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 23:45:47.267665 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2091c6a4-36dc-430c-9288-230d62d653c3 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:47.288993 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2091c6a4-36dc-430c-9288-230d62d653c3 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:47.296779 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77be4f23-580a-484a-818f-aaab771538b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:47.297377 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77be4f23-580a-484a-818f-aaab771538b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:47.297663 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77be4f23-580a-484a-818f-aaab771538b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:47.297944 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77be4f23-580a-484a-818f-aaab771538b8 admin admin] PATH_INFO: `/v3/projects/9f504189f5a949cdae85a009e3b182b1/users/f34ca182eb1c4627beb18ac1256607f8/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:47.313290 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77be4f23-580a-484a-818f-aaab771538b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f504189f5a949cdae85a009e3b182b1, user_id=f34ca182eb1c4627beb18ac1256607f8, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:47.315992 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77be4f23-580a-484a-818f-aaab771538b8 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:47.335190 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1073/2148] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:45:47 2022] PUT /identity/v3/projects/9f504189f5a949cdae85a009e3b182b1/users/f34ca182eb1c4627beb18ac1256607f8/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:45:47.346206 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c28be-7f71-488f-8331-efe814e2c225 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:47.346490 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c28be-7f71-488f-8331-efe814e2c225 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:47.346763 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c28be-7f71-488f-8331-efe814e2c225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:47.375036 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-524c28be-7f71-488f-8331-efe814e2c225 None None] MFA Rules not processed for user `f34ca182eb1c4627beb18ac1256607f8`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:47.377978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-524c28be-7f71-488f-8331-efe814e2c225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:47.456511 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1076/2149] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:45:47.468024 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77be4f23-580a-484a-818f-aaab771538b8 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:45:47.469887 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-77be4f23-580a-484a-818f-aaab771538b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:47.530917 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77be4f23-580a-484a-818f-aaab771538b8 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:45:47.534676 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8c9b9f-9887-4db5-8f37-206e2e3e2051 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:45:47.535297 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c9b9f-9887-4db5-8f37-206e2e3e2051 service cinder] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:47.535841 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c9b9f-9887-4db5-8f37-206e2e3e2051 service cinder] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:47.536159 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c9b9f-9887-4db5-8f37-206e2e3e2051 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:47.548016 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8c9b9f-9887-4db5-8f37-206e2e3e2051 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:45:47.549447 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8c9b9f-9887-4db5-8f37-206e2e3e2051 service cinder] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:45:47.559122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1074/2150] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:45:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:45:49.984022 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38937e3-70e0-4c81-85ba-2ada03389946 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:45:49.984496 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38937e3-70e0-4c81-85ba-2ada03389946 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:45:49.984590 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38937e3-70e0-4c81-85ba-2ada03389946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:45:50.003415 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-f38937e3-70e0-4c81-85ba-2ada03389946 None None] MFA Rules not processed for user `f34ca182eb1c4627beb18ac1256607f8`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:45:50.004766 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-f38937e3-70e0-4c81-85ba-2ada03389946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:45:50.015877 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1077/2151] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:45:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:45:57.554615 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:DELETE=4,keystone:SELECT=60,keystone:INSERT=3 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:45:59.998779 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=44,keystone:INSERT=3 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:46:00.276947 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8c9b9f-9887-4db5-8f37-206e2e3e2051 service cinder] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:00.278375 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-da8c9b9f-9887-4db5-8f37-206e2e3e2051 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:00.344654 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8c9b9f-9887-4db5-8f37-206e2e3e2051 service cinder] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:00.349123 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b3a27-d488-4726-8d4d-f4f5b1abe39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:00.349739 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b3a27-d488-4726-8d4d-f4f5b1abe39c admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:00.350021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b3a27-d488-4726-8d4d-f4f5b1abe39c admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:00.350296 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b3a27-d488-4726-8d4d-f4f5b1abe39c admin admin] PATH_INFO: `/v3/users/3c942b1fcb884310b67c097e6cbfb3d0` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:00.351434 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b3a27-d488-4726-8d4d-f4f5b1abe39c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c942b1fcb884310b67c097e6cbfb3d0)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:00.353420 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b3a27-d488-4726-8d4d-f4f5b1abe39c admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:00.408237 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-c76b3a27-d488-4726-8d4d-f4f5b1abe39c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c942b1fcb884310b67c097e6cbfb3d0'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:00.409196 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-c76b3a27-d488-4726-8d4d-f4f5b1abe39c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c942b1fcb884310b67c097e6cbfb3d0'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:00.409794 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-c76b3a27-d488-4726-8d4d-f4f5b1abe39c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c942b1fcb884310b67c097e6cbfb3d0'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:00.418416 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-c76b3a27-d488-4726-8d4d-f4f5b1abe39c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c942b1fcb884310b67c097e6cbfb3d0'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:00.425797 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-c76b3a27-d488-4726-8d4d-f4f5b1abe39c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c942b1fcb884310b67c097e6cbfb3d0'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:00.449961 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1075/2152] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:46:00 2022] DELETE /identity/v3/users/3c942b1fcb884310b67c097e6cbfb3d0 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:46:00.454419 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2552a0-67ee-4089-872a-846152a30a60 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:00.454705 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2552a0-67ee-4089-872a-846152a30a60 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:00.454967 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2552a0-67ee-4089-872a-846152a30a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:00.475665 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-af2552a0-67ee-4089-872a-846152a30a60 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:46:00.476829 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-af2552a0-67ee-4089-872a-846152a30a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:00.538596 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1078/2153] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:46:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:46:00.548558 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b3a27-d488-4726-8d4d-f4f5b1abe39c admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:00.550013 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-c76b3a27-d488-4726-8d4d-f4f5b1abe39c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:00.607710 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b3a27-d488-4726-8d4d-f4f5b1abe39c admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:00.611344 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f963913c-3af0-4553-8cfa-92915e8e7a7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:00.611993 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963913c-3af0-4553-8cfa-92915e8e7a7a service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:00.612273 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963913c-3af0-4553-8cfa-92915e8e7a7a service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:00.612545 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963913c-3af0-4553-8cfa-92915e8e7a7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:00.621707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f963913c-3af0-4553-8cfa-92915e8e7a7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:00.623131 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f963913c-3af0-4553-8cfa-92915e8e7a7a service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:00.632569 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1076/2154] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:46:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:46:01.277724 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2552a0-67ee-4089-872a-846152a30a60 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:01.279395 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-af2552a0-67ee-4089-872a-846152a30a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:01.295647 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2552a0-67ee-4089-872a-846152a30a60 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:01.315008 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc27d1c-f301-46e5-9fb7-1a8193c9ff30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:01.315680 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc27d1c-f301-46e5-9fb7-1a8193c9ff30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:01.315972 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc27d1c-f301-46e5-9fb7-1a8193c9ff30 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:01.316266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc27d1c-f301-46e5-9fb7-1a8193c9ff30 admin admin] PATH_INFO: `/v3/projects/731f642b2f914908a1e2af1c80462f8f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:01.317763 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc27d1c-f301-46e5-9fb7-1a8193c9ff30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=731f642b2f914908a1e2af1c80462f8f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:01.320811 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc27d1c-f301-46e5-9fb7-1a8193c9ff30 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:01.368637 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-efc27d1c-f301-46e5-9fb7-1a8193c9ff30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '731f642b2f914908a1e2af1c80462f8f', 'request_id': 'req-efc27d1c-f301-46e5-9fb7-1a8193c9ff30'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:01.375208 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-efc27d1c-f301-46e5-9fb7-1a8193c9ff30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '731f642b2f914908a1e2af1c80462f8f', 'request_id': 'req-efc27d1c-f301-46e5-9fb7-1a8193c9ff30'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:01.382684 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-efc27d1c-f301-46e5-9fb7-1a8193c9ff30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '731f642b2f914908a1e2af1c80462f8f', 'request_id': 'req-efc27d1c-f301-46e5-9fb7-1a8193c9ff30'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:01.395490 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-efc27d1c-f301-46e5-9fb7-1a8193c9ff30 admin admin] The token cache is being invalidate because project 731f642b2f914908a1e2af1c80462f8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:46:01.395783 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-efc27d1c-f301-46e5-9fb7-1a8193c9ff30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:01.396533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-efc27d1c-f301-46e5-9fb7-1a8193c9ff30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:01.398231 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1079/2155] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:46:01 2022] DELETE /identity/v3/projects/731f642b2f914908a1e2af1c80462f8f => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:46:01.403115 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6fc18-f1fe-4a7f-97f2-552f1cd7048a None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:01.403609 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6fc18-f1fe-4a7f-97f2-552f1cd7048a None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:01.403701 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6fc18-f1fe-4a7f-97f2-552f1cd7048a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:01.423373 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-fbd6fc18-f1fe-4a7f-97f2-552f1cd7048a None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:46:01.424716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-fbd6fc18-f1fe-4a7f-97f2-552f1cd7048a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:01.443800 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:46:01.490711 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1077/2156] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:46:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:46:01.499495 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fabc83-483a-45ed-a158-f5ff0d59c2cb None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:01.499777 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fabc83-483a-45ed-a158-f5ff0d59c2cb None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:01.500152 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fabc83-483a-45ed-a158-f5ff0d59c2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:01.520530 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-97fabc83-483a-45ed-a158-f5ff0d59c2cb None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:46:01.521723 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-97fabc83-483a-45ed-a158-f5ff0d59c2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:01.529796 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1080/2157] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:46:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:46:01.534102 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd6fc18-f1fe-4a7f-97f2-552f1cd7048a None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:01.544896 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd6fc18-f1fe-4a7f-97f2-552f1cd7048a None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:01.549671 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e33f693-fc94-4669-9bfa-4d11f467c87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:01.550275 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33f693-fc94-4669-9bfa-4d11f467c87a admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:01.550558 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33f693-fc94-4669-9bfa-4d11f467c87a admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:01.550832 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33f693-fc94-4669-9bfa-4d11f467c87a admin admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:01.551484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e33f693-fc94-4669-9bfa-4d11f467c87a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:46:01.551788 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e33f693-fc94-4669-9bfa-4d11f467c87a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:01.553416 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e33f693-fc94-4669-9bfa-4d11f467c87a admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:01.565869 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1078/2158] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:46:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:46:01.570012 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fabc83-483a-45ed-a158-f5ff0d59c2cb None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:01.579466 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fabc83-483a-45ed-a158-f5ff0d59c2cb None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:01.584611 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752ff11-3ab4-4324-8359-5b8b18f91532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:01.585206 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752ff11-3ab4-4324-8359-5b8b18f91532 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:01.585488 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752ff11-3ab4-4324-8359-5b8b18f91532 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:01.585763 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752ff11-3ab4-4324-8359-5b8b18f91532 admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:01.586403 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a752ff11-3ab4-4324-8359-5b8b18f91532 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:01.588395 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a752ff11-3ab4-4324-8359-5b8b18f91532 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:01.622123 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1081/2159] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:46:01 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:46:01.623448 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e33f693-fc94-4669-9bfa-4d11f467c87a admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:01.633977 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e33f693-fc94-4669-9bfa-4d11f467c87a admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:01.694142 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f2124a-2ec0-44ad-891a-0a195019a5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:01.694759 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2124a-2ec0-44ad-891a-0a195019a5f4 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:01.695038 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2124a-2ec0-44ad-891a-0a195019a5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:01.695308 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2124a-2ec0-44ad-891a-0a195019a5f4 admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:01.695941 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f2124a-2ec0-44ad-891a-0a195019a5f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:01.697919 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f2124a-2ec0-44ad-891a-0a195019a5f4 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:01.723753 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1079/2160] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:46:01 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:46:01.724992 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752ff11-3ab4-4324-8359-5b8b18f91532 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:01.735780 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752ff11-3ab4-4324-8359-5b8b18f91532 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:01.740415 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4244977e-20d0-4217-b46b-faa131aebde3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:01.741011 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244977e-20d0-4217-b46b-faa131aebde3 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:01.741285 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244977e-20d0-4217-b46b-faa131aebde3 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:01.741550 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244977e-20d0-4217-b46b-faa131aebde3 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:01.742222 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4244977e-20d0-4217-b46b-faa131aebde3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:01.743835 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4244977e-20d0-4217-b46b-faa131aebde3 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:01.757824 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1082/2161] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:46:01.759011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f2124a-2ec0-44ad-891a-0a195019a5f4 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:01.769132 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f2124a-2ec0-44ad-891a-0a195019a5f4 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:01.774036 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e2dd90-6edc-41dc-a0aa-f62f1937b58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:01.774589 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2dd90-6edc-41dc-a0aa-f62f1937b58e admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:01.774868 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2dd90-6edc-41dc-a0aa-f62f1937b58e admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:01.775142 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2dd90-6edc-41dc-a0aa-f62f1937b58e admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:01.775792 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e2dd90-6edc-41dc-a0aa-f62f1937b58e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:01.777470 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e2dd90-6edc-41dc-a0aa-f62f1937b58e admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:01.788408 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1080/2162] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:46:01.793045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4244977e-20d0-4217-b46b-faa131aebde3 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:01.804510 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4244977e-20d0-4217-b46b-faa131aebde3 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:01.810021 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5da57f5-27ab-4e44-b189-3fdf463fa5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:01.810671 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da57f5-27ab-4e44-b189-3fdf463fa5e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:01.810950 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da57f5-27ab-4e44-b189-3fdf463fa5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:01.811249 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da57f5-27ab-4e44-b189-3fdf463fa5e4 admin admin] PATH_INFO: `/v3/projects/ad1938cc2d00407f968656ebe9e57b85/users/fb514294934646e0a0127466a612057a/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:01.827290 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5da57f5-27ab-4e44-b189-3fdf463fa5e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad1938cc2d00407f968656ebe9e57b85, user_id=fb514294934646e0a0127466a612057a, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:01.829994 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5da57f5-27ab-4e44-b189-3fdf463fa5e4 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:01.845521 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1083/2163] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:46:01 2022] PUT /identity/v3/projects/ad1938cc2d00407f968656ebe9e57b85/users/fb514294934646e0a0127466a612057a/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:46:01.856451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bbb6aa-5147-48c6-92c3-a43243a17e2f None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:01.856746 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bbb6aa-5147-48c6-92c3-a43243a17e2f None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:01.857027 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bbb6aa-5147-48c6-92c3-a43243a17e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:01.884787 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-d6bbb6aa-5147-48c6-92c3-a43243a17e2f None None] MFA Rules not processed for user `fb514294934646e0a0127466a612057a`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:46:01.886079 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-d6bbb6aa-5147-48c6-92c3-a43243a17e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:01.951495 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1081/2164] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:46:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:46:01.956162 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5da57f5-27ab-4e44-b189-3fdf463fa5e4 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:01.977212 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:46:02.022404 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5da57f5-27ab-4e44-b189-3fdf463fa5e4 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:02.027624 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45bd585-a47f-4fe5-8f0d-83b4a947eb0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:02.028183 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45bd585-a47f-4fe5-8f0d-83b4a947eb0e admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:02.028460 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45bd585-a47f-4fe5-8f0d-83b4a947eb0e admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:02.029135 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45bd585-a47f-4fe5-8f0d-83b4a947eb0e admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:02.029331 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45bd585-a47f-4fe5-8f0d-83b4a947eb0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:02.031228 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45bd585-a47f-4fe5-8f0d-83b4a947eb0e admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:02.062314 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1084/2165] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:46:01 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:46:02.063458 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bbb6aa-5147-48c6-92c3-a43243a17e2f None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:02.074165 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bbb6aa-5147-48c6-92c3-a43243a17e2f None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:02.087208 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:46:02.133036 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ecea0-fa2a-4f4e-8cb5-99fbb8c600b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:02.133757 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ecea0-fa2a-4f4e-8cb5-99fbb8c600b9 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:02.134115 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ecea0-fa2a-4f4e-8cb5-99fbb8c600b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:02.134397 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ecea0-fa2a-4f4e-8cb5-99fbb8c600b9 admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:02.135035 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ecea0-fa2a-4f4e-8cb5-99fbb8c600b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:02.137075 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ecea0-fa2a-4f4e-8cb5-99fbb8c600b9 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:02.164838 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1082/2166] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:46:02 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:46:02.166052 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45bd585-a47f-4fe5-8f0d-83b4a947eb0e admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:02.179329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45bd585-a47f-4fe5-8f0d-83b4a947eb0e admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:02.183811 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144fa093-9fdf-4f04-b347-f5d77bdaa31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:02.184471 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144fa093-9fdf-4f04-b347-f5d77bdaa31d admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:02.184753 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144fa093-9fdf-4f04-b347-f5d77bdaa31d admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:02.185168 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144fa093-9fdf-4f04-b347-f5d77bdaa31d admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:02.185889 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144fa093-9fdf-4f04-b347-f5d77bdaa31d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:02.187768 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144fa093-9fdf-4f04-b347-f5d77bdaa31d admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:02.200365 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1085/2167] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:46:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:46:02.204232 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ecea0-fa2a-4f4e-8cb5-99fbb8c600b9 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:02.219528 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ecea0-fa2a-4f4e-8cb5-99fbb8c600b9 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:02.226856 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b3516-6371-483c-b2b8-9c40ce024ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:02.227480 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b3516-6371-483c-b2b8-9c40ce024ccd admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:02.227763 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b3516-6371-483c-b2b8-9c40ce024ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:02.228077 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b3516-6371-483c-b2b8-9c40ce024ccd admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:02.228737 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1b3516-6371-483c-b2b8-9c40ce024ccd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:02.231041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1b3516-6371-483c-b2b8-9c40ce024ccd admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:02.245456 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1083/2168] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:46:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:46:02.246617 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144fa093-9fdf-4f04-b347-f5d77bdaa31d admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:02.259446 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144fa093-9fdf-4f04-b347-f5d77bdaa31d admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:02.267116 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44758657-3eb6-4f4c-a3ec-d3749546d9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:02.268642 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44758657-3eb6-4f4c-a3ec-d3749546d9a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:02.269098 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44758657-3eb6-4f4c-a3ec-d3749546d9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:02.269523 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44758657-3eb6-4f4c-a3ec-d3749546d9a6 admin admin] PATH_INFO: `/v3/projects/f6799a3443f4467d937fc47114dd4bd0/users/5f42e2209ca5440dbd62f07ebe6960fa/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:02.291361 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44758657-3eb6-4f4c-a3ec-d3749546d9a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6799a3443f4467d937fc47114dd4bd0, user_id=5f42e2209ca5440dbd62f07ebe6960fa, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:02.294433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44758657-3eb6-4f4c-a3ec-d3749546d9a6 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:02.309614 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1086/2169] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:46:02 2022] PUT /identity/v3/projects/f6799a3443f4467d937fc47114dd4bd0/users/5f42e2209ca5440dbd62f07ebe6960fa/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:46:02.321092 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e277cd-4793-4e59-9879-940b49979d13 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:02.321394 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e277cd-4793-4e59-9879-940b49979d13 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:02.321820 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e277cd-4793-4e59-9879-940b49979d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:02.356271 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-43e277cd-4793-4e59-9879-940b49979d13 None None] MFA Rules not processed for user `5f42e2209ca5440dbd62f07ebe6960fa`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:46:02.357569 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-43e277cd-4793-4e59-9879-940b49979d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:02.448114 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1084/2170] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:46:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:46:02.456439 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44758657-3eb6-4f4c-a3ec-d3749546d9a6 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:02.532646 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44758657-3eb6-4f4c-a3ec-d3749546d9a6 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:02.537924 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a022179-4a2a-426a-a29d-97cd73188552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:02.538624 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a022179-4a2a-426a-a29d-97cd73188552 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:02.538734 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a022179-4a2a-426a-a29d-97cd73188552 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:02.539166 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a022179-4a2a-426a-a29d-97cd73188552 admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:02.539941 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a022179-4a2a-426a-a29d-97cd73188552 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:02.541989 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a022179-4a2a-426a-a29d-97cd73188552 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:02.569209 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1087/2171] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:46:02 2022] POST /identity/v3/projects => generated 377 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:46:02.573283 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e277cd-4793-4e59-9879-940b49979d13 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:02.584642 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e277cd-4793-4e59-9879-940b49979d13 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:02.646695 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6a4b5-cba3-473d-b6ad-825b69cc7a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:02.647307 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6a4b5-cba3-473d-b6ad-825b69cc7a3d admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:02.647588 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6a4b5-cba3-473d-b6ad-825b69cc7a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:02.647860 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6a4b5-cba3-473d-b6ad-825b69cc7a3d admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:02.648513 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f6a4b5-cba3-473d-b6ad-825b69cc7a3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:02.650455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f6a4b5-cba3-473d-b6ad-825b69cc7a3d admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:02.674951 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1085/2172] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:46:02 2022] POST /identity/v3/users => generated 295 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:46:02.678942 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a022179-4a2a-426a-a29d-97cd73188552 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:02.691215 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a022179-4a2a-426a-a29d-97cd73188552 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:02.696139 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f772f02-cca9-47c0-af83-34b621794626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:02.696740 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f772f02-cca9-47c0-af83-34b621794626 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:02.697018 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f772f02-cca9-47c0-af83-34b621794626 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:02.697287 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f772f02-cca9-47c0-af83-34b621794626 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:02.697933 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f772f02-cca9-47c0-af83-34b621794626 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:02.700182 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f772f02-cca9-47c0-af83-34b621794626 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:02.711011 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1088/2173] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:46:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:46:02.715301 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6a4b5-cba3-473d-b6ad-825b69cc7a3d admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:02.725156 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6a4b5-cba3-473d-b6ad-825b69cc7a3d admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:02.729775 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257b5a5-c039-457e-8d07-1c2a5ffca241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:02.730426 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257b5a5-c039-457e-8d07-1c2a5ffca241 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:02.730703 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257b5a5-c039-457e-8d07-1c2a5ffca241 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:02.730973 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257b5a5-c039-457e-8d07-1c2a5ffca241 admin admin] PATH_INFO: `/v3/projects/10399a5f99cf40009d996b664c303a78/users/2ced474208f845c4b36dbd25dbfd1b76/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:02.743211 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:46:02.747815 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257b5a5-c039-457e-8d07-1c2a5ffca241 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10399a5f99cf40009d996b664c303a78, user_id=2ced474208f845c4b36dbd25dbfd1b76, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:02.751052 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257b5a5-c039-457e-8d07-1c2a5ffca241 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:02.765385 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1086/2174] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:46:02 2022] PUT /identity/v3/projects/10399a5f99cf40009d996b664c303a78/users/2ced474208f845c4b36dbd25dbfd1b76/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:46:02.769374 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f772f02-cca9-47c0-af83-34b621794626 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:02.779370 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f772f02-cca9-47c0-af83-34b621794626 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:02.823713 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:46:02.840704 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b80f4-1bdf-4bb9-a9ba-46ea208c84f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:02.841326 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b80f4-1bdf-4bb9-a9ba-46ea208c84f9 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:02.841605 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b80f4-1bdf-4bb9-a9ba-46ea208c84f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:02.841875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b80f4-1bdf-4bb9-a9ba-46ea208c84f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:02.842523 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785b80f4-1bdf-4bb9-a9ba-46ea208c84f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:02.844404 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785b80f4-1bdf-4bb9-a9ba-46ea208c84f9 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:02.858737 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1089/2175] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:46:02 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:46:02.860044 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257b5a5-c039-457e-8d07-1c2a5ffca241 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:02.870771 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257b5a5-c039-457e-8d07-1c2a5ffca241 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:02.875563 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc6f1c-8889-44fa-b596-a8409808daad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:02.876478 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc6f1c-8889-44fa-b596-a8409808daad admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:02.876853 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc6f1c-8889-44fa-b596-a8409808daad admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:02.877304 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc6f1c-8889-44fa-b596-a8409808daad admin admin] PATH_INFO: `/v3/projects/10399a5f99cf40009d996b664c303a78/users/2ced474208f845c4b36dbd25dbfd1b76/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:02.879321 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cc6f1c-8889-44fa-b596-a8409808daad admin admin] RBAC: Authorizing `identity:create_grant(project_id=10399a5f99cf40009d996b664c303a78, user_id=2ced474208f845c4b36dbd25dbfd1b76, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:02.882105 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cc6f1c-8889-44fa-b596-a8409808daad admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:02.893412 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1087/2176] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:46:02 2022] PUT /identity/v3/projects/10399a5f99cf40009d996b664c303a78/users/2ced474208f845c4b36dbd25dbfd1b76/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:46:02.904329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f3d8f-6388-4035-b2d6-eca9911629d2 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:02.904614 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f3d8f-6388-4035-b2d6-eca9911629d2 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:02.904888 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f3d8f-6388-4035-b2d6-eca9911629d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:02.934341 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-8b6f3d8f-6388-4035-b2d6-eca9911629d2 None None] MFA Rules not processed for user `2ced474208f845c4b36dbd25dbfd1b76`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:46:02.935547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-8b6f3d8f-6388-4035-b2d6-eca9911629d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:03.003096 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1090/2177] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:46:02 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:46:03.014887 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc6f1c-8889-44fa-b596-a8409808daad admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:03.016679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-88cc6f1c-8889-44fa-b596-a8409808daad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:03.074997 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc6f1c-8889-44fa-b596-a8409808daad admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:03.078597 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e698e0b8-7fd6-4efc-bf1b-8d8dca4e595f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:03.079382 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698e0b8-7fd6-4efc-bf1b-8d8dca4e595f service cinder] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:03.079670 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698e0b8-7fd6-4efc-bf1b-8d8dca4e595f service cinder] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:03.079943 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698e0b8-7fd6-4efc-bf1b-8d8dca4e595f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:03.139493 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e698e0b8-7fd6-4efc-bf1b-8d8dca4e595f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:03.141066 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e698e0b8-7fd6-4efc-bf1b-8d8dca4e595f service cinder] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:03.150558 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1088/2178] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:46:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:46:05.740522 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6f3d8f-6388-4035-b2d6-eca9911629d2 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:05.750594 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6f3d8f-6388-4035-b2d6-eca9911629d2 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:05.755331 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c140fe1-9f5e-42ac-97ab-322d795db8fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:05.756150 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c140fe1-9f5e-42ac-97ab-322d795db8fb service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:05.756271 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c140fe1-9f5e-42ac-97ab-322d795db8fb service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:05.756503 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c140fe1-9f5e-42ac-97ab-322d795db8fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:05.830599 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c140fe1-9f5e-42ac-97ab-322d795db8fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:05.832422 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c140fe1-9f5e-42ac-97ab-322d795db8fb service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:05.864251 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1091/2179] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:46:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:46:06.461623 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e698e0b8-7fd6-4efc-bf1b-8d8dca4e595f service cinder] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:06.463923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-e698e0b8-7fd6-4efc-bf1b-8d8dca4e595f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:06.476481 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e698e0b8-7fd6-4efc-bf1b-8d8dca4e595f service cinder] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:06.480648 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c390f3b-785a-41d6-abeb-35754ec70056 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:06.481254 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c390f3b-785a-41d6-abeb-35754ec70056 service cinder] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:06.481536 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c390f3b-785a-41d6-abeb-35754ec70056 service cinder] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:06.481809 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c390f3b-785a-41d6-abeb-35754ec70056 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:06.502864 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c390f3b-785a-41d6-abeb-35754ec70056 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:06.502864 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c390f3b-785a-41d6-abeb-35754ec70056 service cinder] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:06.509224 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1089/2180] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:46:06 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:46:10.417966 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c140fe1-9f5e-42ac-97ab-322d795db8fb service cinder] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:10.480817 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c140fe1-9f5e-42ac-97ab-322d795db8fb service cinder] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:10.485315 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef538de-5609-4c15-811f-dce8576007d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:10.485939 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef538de-5609-4c15-811f-dce8576007d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:10.486212 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef538de-5609-4c15-811f-dce8576007d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:10.486483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef538de-5609-4c15-811f-dce8576007d6 admin admin] PATH_INFO: `/v3/users/fb514294934646e0a0127466a612057a` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:10.487588 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef538de-5609-4c15-811f-dce8576007d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb514294934646e0a0127466a612057a)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:10.489540 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef538de-5609-4c15-811f-dce8576007d6 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:10.493990 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:46:10.536270 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-fef538de-5609-4c15-811f-dce8576007d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb514294934646e0a0127466a612057a'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:10.536917 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-fef538de-5609-4c15-811f-dce8576007d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb514294934646e0a0127466a612057a'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:10.548916 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-fef538de-5609-4c15-811f-dce8576007d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb514294934646e0a0127466a612057a'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:10.555991 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-fef538de-5609-4c15-811f-dce8576007d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb514294934646e0a0127466a612057a'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:10.556600 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-fef538de-5609-4c15-811f-dce8576007d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb514294934646e0a0127466a612057a'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:10.568687 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1092/2181] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:46:10 2022] DELETE /identity/v3/users/fb514294934646e0a0127466a612057a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:46:10.573603 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f887716-95ce-45ea-895d-b315e2229371 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:10.574066 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f887716-95ce-45ea-895d-b315e2229371 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:10.574162 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f887716-95ce-45ea-895d-b315e2229371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:10.592812 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-2f887716-95ce-45ea-895d-b315e2229371 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:46:10.594004 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-2f887716-95ce-45ea-895d-b315e2229371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:10.652451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1090/2182] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:46:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:46:10.663546 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef538de-5609-4c15-811f-dce8576007d6 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:10.664946 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-fef538de-5609-4c15-811f-dce8576007d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:10.724203 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef538de-5609-4c15-811f-dce8576007d6 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:10.729011 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b1fc27-803c-4661-b9d6-0c87d82500a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:10.729011 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b1fc27-803c-4661-b9d6-0c87d82500a4 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:10.729246 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b1fc27-803c-4661-b9d6-0c87d82500a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:10.729429 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b1fc27-803c-4661-b9d6-0c87d82500a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:10.739774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b1fc27-803c-4661-b9d6-0c87d82500a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:10.741288 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b1fc27-803c-4661-b9d6-0c87d82500a4 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:10.751567 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1093/2183] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:46:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:46:11.063992 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f887716-95ce-45ea-895d-b315e2229371 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:11.065502 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-2f887716-95ce-45ea-895d-b315e2229371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:11.095645 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f887716-95ce-45ea-895d-b315e2229371 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:11.102130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b30cee-c19d-49fa-bae0-b50e13c63c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:11.102896 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b30cee-c19d-49fa-bae0-b50e13c63c92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:11.103418 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b30cee-c19d-49fa-bae0-b50e13c63c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:11.103794 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b30cee-c19d-49fa-bae0-b50e13c63c92 admin admin] PATH_INFO: `/v3/projects/ad1938cc2d00407f968656ebe9e57b85` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:11.105323 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b30cee-c19d-49fa-bae0-b50e13c63c92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad1938cc2d00407f968656ebe9e57b85)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:11.108146 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b30cee-c19d-49fa-bae0-b50e13c63c92 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:11.126588 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:46:11.158365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-a7b30cee-c19d-49fa-bae0-b50e13c63c92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad1938cc2d00407f968656ebe9e57b85', 'request_id': 'req-a7b30cee-c19d-49fa-bae0-b50e13c63c92'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:11.164338 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-a7b30cee-c19d-49fa-bae0-b50e13c63c92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad1938cc2d00407f968656ebe9e57b85', 'request_id': 'req-a7b30cee-c19d-49fa-bae0-b50e13c63c92'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:11.171440 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-a7b30cee-c19d-49fa-bae0-b50e13c63c92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad1938cc2d00407f968656ebe9e57b85', 'request_id': 'req-a7b30cee-c19d-49fa-bae0-b50e13c63c92'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:11.182127 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-a7b30cee-c19d-49fa-bae0-b50e13c63c92 admin admin] The token cache is being invalidate because project ad1938cc2d00407f968656ebe9e57b85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:46:11.182415 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-a7b30cee-c19d-49fa-bae0-b50e13c63c92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:11.183559 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-a7b30cee-c19d-49fa-bae0-b50e13c63c92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:11.185093 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1091/2184] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:46:11 2022] DELETE /identity/v3/projects/ad1938cc2d00407f968656ebe9e57b85 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:46:11.189151 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b1fc27-803c-4661-b9d6-0c87d82500a4 service neutron] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:11.199695 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b1fc27-803c-4661-b9d6-0c87d82500a4 service neutron] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:11.253418 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f76fd-52b8-4ad8-817a-a2c49fafb05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:11.254028 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f76fd-52b8-4ad8-817a-a2c49fafb05c admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:11.254303 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f76fd-52b8-4ad8-817a-a2c49fafb05c admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:11.254570 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f76fd-52b8-4ad8-817a-a2c49fafb05c admin admin] PATH_INFO: `/v3/users/5f42e2209ca5440dbd62f07ebe6960fa` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:11.255656 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311f76fd-52b8-4ad8-817a-a2c49fafb05c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f42e2209ca5440dbd62f07ebe6960fa)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:11.257673 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311f76fd-52b8-4ad8-817a-a2c49fafb05c admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:11.286917 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:46:11.305517 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-311f76fd-52b8-4ad8-817a-a2c49fafb05c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f42e2209ca5440dbd62f07ebe6960fa'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:11.306162 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-311f76fd-52b8-4ad8-817a-a2c49fafb05c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f42e2209ca5440dbd62f07ebe6960fa'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:11.318504 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-311f76fd-52b8-4ad8-817a-a2c49fafb05c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f42e2209ca5440dbd62f07ebe6960fa'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:11.324977 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-311f76fd-52b8-4ad8-817a-a2c49fafb05c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f42e2209ca5440dbd62f07ebe6960fa'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:11.325777 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-311f76fd-52b8-4ad8-817a-a2c49fafb05c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f42e2209ca5440dbd62f07ebe6960fa'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:11.337813 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1094/2185] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:46:11 2022] DELETE /identity/v3/users/5f42e2209ca5440dbd62f07ebe6960fa => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:46:11.613073 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b30cee-c19d-49fa-bae0-b50e13c63c92 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:11.614608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-a7b30cee-c19d-49fa-bae0-b50e13c63c92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:11.629746 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b30cee-c19d-49fa-bae0-b50e13c63c92 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:11.693896 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a64edc-2486-40f7-b51c-ae4bd28ca2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:11.694460 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a64edc-2486-40f7-b51c-ae4bd28ca2b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:11.694747 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a64edc-2486-40f7-b51c-ae4bd28ca2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:11.695022 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a64edc-2486-40f7-b51c-ae4bd28ca2b7 admin admin] PATH_INFO: `/v3/projects/f6799a3443f4467d937fc47114dd4bd0` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:11.696183 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a64edc-2486-40f7-b51c-ae4bd28ca2b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6799a3443f4467d937fc47114dd4bd0)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:11.698740 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a64edc-2486-40f7-b51c-ae4bd28ca2b7 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:11.742183 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-f3a64edc-2486-40f7-b51c-ae4bd28ca2b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6799a3443f4467d937fc47114dd4bd0', 'request_id': 'req-f3a64edc-2486-40f7-b51c-ae4bd28ca2b7'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:11.748078 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-f3a64edc-2486-40f7-b51c-ae4bd28ca2b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6799a3443f4467d937fc47114dd4bd0', 'request_id': 'req-f3a64edc-2486-40f7-b51c-ae4bd28ca2b7'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:11.755021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-f3a64edc-2486-40f7-b51c-ae4bd28ca2b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6799a3443f4467d937fc47114dd4bd0', 'request_id': 'req-f3a64edc-2486-40f7-b51c-ae4bd28ca2b7'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:11.766514 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-f3a64edc-2486-40f7-b51c-ae4bd28ca2b7 admin admin] The token cache is being invalidate because project f6799a3443f4467d937fc47114dd4bd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:46:11.766810 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-f3a64edc-2486-40f7-b51c-ae4bd28ca2b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:11.767445 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-f3a64edc-2486-40f7-b51c-ae4bd28ca2b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:11.769035 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1092/2186] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:46:11 2022] DELETE /identity/v3/projects/f6799a3443f4467d937fc47114dd4bd0 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:46:11.773023 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f76fd-52b8-4ad8-817a-a2c49fafb05c admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:11.774457 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-311f76fd-52b8-4ad8-817a-a2c49fafb05c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:11.787387 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f76fd-52b8-4ad8-817a-a2c49fafb05c admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:11.845009 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544a1ea-4372-4b30-adc2-9631ce45d3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:11.845628 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544a1ea-4372-4b30-adc2-9631ce45d3c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:11.845907 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544a1ea-4372-4b30-adc2-9631ce45d3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:11.846176 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544a1ea-4372-4b30-adc2-9631ce45d3c6 admin admin] PATH_INFO: `/v3/users/2ced474208f845c4b36dbd25dbfd1b76` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:11.847311 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b544a1ea-4372-4b30-adc2-9631ce45d3c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ced474208f845c4b36dbd25dbfd1b76)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:11.849259 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b544a1ea-4372-4b30-adc2-9631ce45d3c6 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:11.894668 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-b544a1ea-4372-4b30-adc2-9631ce45d3c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ced474208f845c4b36dbd25dbfd1b76'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:11.895314 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-b544a1ea-4372-4b30-adc2-9631ce45d3c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ced474208f845c4b36dbd25dbfd1b76'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:11.907744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-b544a1ea-4372-4b30-adc2-9631ce45d3c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ced474208f845c4b36dbd25dbfd1b76'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:11.915633 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-b544a1ea-4372-4b30-adc2-9631ce45d3c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ced474208f845c4b36dbd25dbfd1b76'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:11.916269 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-b544a1ea-4372-4b30-adc2-9631ce45d3c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ced474208f845c4b36dbd25dbfd1b76'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:11.929074 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1095/2187] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:46:11 2022] DELETE /identity/v3/users/2ced474208f845c4b36dbd25dbfd1b76 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:46:12.201019 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a64edc-2486-40f7-b51c-ae4bd28ca2b7 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:12.202500 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-f3a64edc-2486-40f7-b51c-ae4bd28ca2b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:12.216888 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a64edc-2486-40f7-b51c-ae4bd28ca2b7 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:12.247631 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:46:12.274816 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c6e91e-f957-47ef-8789-fc9b8c1a03cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:12.275435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c6e91e-f957-47ef-8789-fc9b8c1a03cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:12.275711 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c6e91e-f957-47ef-8789-fc9b8c1a03cf admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:12.276008 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c6e91e-f957-47ef-8789-fc9b8c1a03cf admin admin] PATH_INFO: `/v3/projects/10399a5f99cf40009d996b664c303a78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:12.277383 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c6e91e-f957-47ef-8789-fc9b8c1a03cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=10399a5f99cf40009d996b664c303a78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:12.279280 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c6e91e-f957-47ef-8789-fc9b8c1a03cf admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:12.325445 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-67c6e91e-f957-47ef-8789-fc9b8c1a03cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10399a5f99cf40009d996b664c303a78', 'request_id': 'req-67c6e91e-f957-47ef-8789-fc9b8c1a03cf'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:12.330875 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-67c6e91e-f957-47ef-8789-fc9b8c1a03cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10399a5f99cf40009d996b664c303a78', 'request_id': 'req-67c6e91e-f957-47ef-8789-fc9b8c1a03cf'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:12.337716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-67c6e91e-f957-47ef-8789-fc9b8c1a03cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10399a5f99cf40009d996b664c303a78', 'request_id': 'req-67c6e91e-f957-47ef-8789-fc9b8c1a03cf'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:12.348781 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-67c6e91e-f957-47ef-8789-fc9b8c1a03cf admin admin] The token cache is being invalidate because project 10399a5f99cf40009d996b664c303a78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:46:12.349066 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-67c6e91e-f957-47ef-8789-fc9b8c1a03cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:12.349695 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-67c6e91e-f957-47ef-8789-fc9b8c1a03cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:12.351148 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1093/2188] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:46:12 2022] DELETE /identity/v3/projects/10399a5f99cf40009d996b664c303a78 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:46:12.356599 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc89f85-3e72-4db0-a0aa-ffe930d702da None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:12.356936 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc89f85-3e72-4db0-a0aa-ffe930d702da None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:12.357209 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc89f85-3e72-4db0-a0aa-ffe930d702da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:12.377084 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-7cc89f85-3e72-4db0-a0aa-ffe930d702da None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:46:12.378289 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-7cc89f85-3e72-4db0-a0aa-ffe930d702da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:12.397105 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:46:12.440177 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1096/2189] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:46:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:46:12.449110 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946cabec-f5c4-46b2-996d-4bd06341c582 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:12.449496 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946cabec-f5c4-46b2-996d-4bd06341c582 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:12.449905 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946cabec-f5c4-46b2-996d-4bd06341c582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:12.471508 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-946cabec-f5c4-46b2-996d-4bd06341c582 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:46:12.472819 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-946cabec-f5c4-46b2-996d-4bd06341c582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:12.481447 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1094/2190] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:46:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:46:12.485806 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc89f85-3e72-4db0-a0aa-ffe930d702da None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:12.495431 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc89f85-3e72-4db0-a0aa-ffe930d702da None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:12.499943 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8efdd-6eac-4b90-82d7-a5f26adad850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:12.500558 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8efdd-6eac-4b90-82d7-a5f26adad850 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:12.500832 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8efdd-6eac-4b90-82d7-a5f26adad850 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:12.501099 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8efdd-6eac-4b90-82d7-a5f26adad850 admin admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:12.501754 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8efdd-6eac-4b90-82d7-a5f26adad850 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:46:12.502037 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8efdd-6eac-4b90-82d7-a5f26adad850 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:12.503639 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8efdd-6eac-4b90-82d7-a5f26adad850 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:12.514820 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1097/2191] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:46:12.518862 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946cabec-f5c4-46b2-996d-4bd06341c582 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:12.528557 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946cabec-f5c4-46b2-996d-4bd06341c582 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:12.533103 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e9612e-1986-4e05-95c1-4590d22eb63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:12.533690 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9612e-1986-4e05-95c1-4590d22eb63a admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:12.533967 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9612e-1986-4e05-95c1-4590d22eb63a admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:12.534238 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9612e-1986-4e05-95c1-4590d22eb63a admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:12.534864 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e9612e-1986-4e05-95c1-4590d22eb63a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:12.537047 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e9612e-1986-4e05-95c1-4590d22eb63a admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:12.563729 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1095/2192] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:46:12 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:46:12.567787 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8efdd-6eac-4b90-82d7-a5f26adad850 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:12.578126 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8efdd-6eac-4b90-82d7-a5f26adad850 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:12.634118 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534940ca-e1a6-4270-9070-f76405fc8f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:12.634705 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534940ca-e1a6-4270-9070-f76405fc8f7b admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:12.634979 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534940ca-e1a6-4270-9070-f76405fc8f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:12.635246 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534940ca-e1a6-4270-9070-f76405fc8f7b admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:12.635853 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534940ca-e1a6-4270-9070-f76405fc8f7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:12.637805 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534940ca-e1a6-4270-9070-f76405fc8f7b admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:12.657953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1098/2193] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:46:12 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:46:12.662253 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e9612e-1986-4e05-95c1-4590d22eb63a admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:12.672994 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e9612e-1986-4e05-95c1-4590d22eb63a admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:12.677576 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88177986-4354-4dcb-abd5-fa6785b43b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:12.678161 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88177986-4354-4dcb-abd5-fa6785b43b4a admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:12.678439 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88177986-4354-4dcb-abd5-fa6785b43b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:12.678708 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88177986-4354-4dcb-abd5-fa6785b43b4a admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:12.679350 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88177986-4354-4dcb-abd5-fa6785b43b4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:12.680964 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88177986-4354-4dcb-abd5-fa6785b43b4a admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:12.691184 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1096/2194] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:46:12.694992 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534940ca-e1a6-4270-9070-f76405fc8f7b admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:12.705450 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534940ca-e1a6-4270-9070-f76405fc8f7b admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:12.710203 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db5e5f-c7bd-4905-aeca-4cf5fe93d489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:12.710798 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db5e5f-c7bd-4905-aeca-4cf5fe93d489 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:12.711078 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db5e5f-c7bd-4905-aeca-4cf5fe93d489 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:12.711350 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db5e5f-c7bd-4905-aeca-4cf5fe93d489 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:12.712170 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8db5e5f-c7bd-4905-aeca-4cf5fe93d489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:12.714273 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8db5e5f-c7bd-4905-aeca-4cf5fe93d489 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:12.724863 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1099/2195] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:46:12.728909 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88177986-4354-4dcb-abd5-fa6785b43b4a admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:12.739414 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88177986-4354-4dcb-abd5-fa6785b43b4a admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:12.744271 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977788f-88ef-4ee8-8000-52d2fc612d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:12.744934 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977788f-88ef-4ee8-8000-52d2fc612d7c admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:12.745216 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977788f-88ef-4ee8-8000-52d2fc612d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:12.745487 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977788f-88ef-4ee8-8000-52d2fc612d7c admin admin] PATH_INFO: `/v3/projects/8e81ed7b06f84b2bab3369334cb1f90a/users/cf6e346e96144628ac25c428be38870a/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:12.759848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b977788f-88ef-4ee8-8000-52d2fc612d7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e81ed7b06f84b2bab3369334cb1f90a, user_id=cf6e346e96144628ac25c428be38870a, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:12.763032 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b977788f-88ef-4ee8-8000-52d2fc612d7c admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:12.775501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1097/2196] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:46:12 2022] PUT /identity/v3/projects/8e81ed7b06f84b2bab3369334cb1f90a/users/cf6e346e96144628ac25c428be38870a/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:46:12.780971 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e765cba-50ca-4b00-819b-9a8d0d3ab20e None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:12.781444 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e765cba-50ca-4b00-819b-9a8d0d3ab20e None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:12.781538 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e765cba-50ca-4b00-819b-9a8d0d3ab20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:12.800280 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-8e765cba-50ca-4b00-819b-9a8d0d3ab20e None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:46:12.801471 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-8e765cba-50ca-4b00-819b-9a8d0d3ab20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:12.858346 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1100/2197] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:46:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:46:12.867804 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977788f-88ef-4ee8-8000-52d2fc612d7c admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:12.869531 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-b977788f-88ef-4ee8-8000-52d2fc612d7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:12.924635 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977788f-88ef-4ee8-8000-52d2fc612d7c admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:12.928217 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3570e671-2f9e-46cb-9d4b-39bc205c48cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:12.928810 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570e671-2f9e-46cb-9d4b-39bc205c48cf service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:12.929088 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570e671-2f9e-46cb-9d4b-39bc205c48cf service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:12.929357 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570e671-2f9e-46cb-9d4b-39bc205c48cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:12.938602 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3570e671-2f9e-46cb-9d4b-39bc205c48cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:12.940045 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3570e671-2f9e-46cb-9d4b-39bc205c48cf service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:12.949135 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1098/2198] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:46:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:46:17.193191 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e210a-d00e-44df-b855-257fe1e47bf2 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:17.194330 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e210a-d00e-44df-b855-257fe1e47bf2 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:17.194559 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e210a-d00e-44df-b855-257fe1e47bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:17.235622 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-032e210a-d00e-44df-b855-257fe1e47bf2 None None] MFA Rules not processed for user `cf6e346e96144628ac25c428be38870a`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:46:17.236877 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-032e210a-d00e-44df-b855-257fe1e47bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:17.282971 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:46:17.393580 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1101/2199] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:46:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:46:17.394863 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3570e671-2f9e-46cb-9d4b-39bc205c48cf service neutron] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:17.400213 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:46:17.407592 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3570e671-2f9e-46cb-9d4b-39bc205c48cf service neutron] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:17.414129 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a23fc-1fc8-4359-9a90-f5d5d49fa919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:17.415483 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a23fc-1fc8-4359-9a90-f5d5d49fa919 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:17.416155 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a23fc-1fc8-4359-9a90-f5d5d49fa919 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:17.417044 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a23fc-1fc8-4359-9a90-f5d5d49fa919 admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:17.418360 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1a23fc-1fc8-4359-9a90-f5d5d49fa919 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:17.421186 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1a23fc-1fc8-4359-9a90-f5d5d49fa919 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:17.459534 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1099/2200] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:46:17 2022] POST /identity/v3/projects => generated 377 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:46:17.463179 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e210a-d00e-44df-b855-257fe1e47bf2 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:17.479684 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e210a-d00e-44df-b855-257fe1e47bf2 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:17.545325 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6e612-feed-46bc-8809-ea0e3ee27b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:17.545939 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6e612-feed-46bc-8809-ea0e3ee27b6f admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:17.546215 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6e612-feed-46bc-8809-ea0e3ee27b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:17.546486 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6e612-feed-46bc-8809-ea0e3ee27b6f admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:17.547120 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a6e612-feed-46bc-8809-ea0e3ee27b6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:17.549456 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a6e612-feed-46bc-8809-ea0e3ee27b6f admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:17.577033 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1102/2201] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:46:17 2022] POST /identity/v3/users => generated 295 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:46:17.581284 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a23fc-1fc8-4359-9a90-f5d5d49fa919 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:17.592807 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a23fc-1fc8-4359-9a90-f5d5d49fa919 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:17.598388 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e21f3-d37a-4687-abc1-0cc97277b8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:17.599008 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e21f3-d37a-4687-abc1-0cc97277b8e6 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:17.599289 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e21f3-d37a-4687-abc1-0cc97277b8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:17.599561 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e21f3-d37a-4687-abc1-0cc97277b8e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:17.600446 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40e21f3-d37a-4687-abc1-0cc97277b8e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:17.602364 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40e21f3-d37a-4687-abc1-0cc97277b8e6 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:17.622849 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1100/2202] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:46:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:46:17.628096 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6e612-feed-46bc-8809-ea0e3ee27b6f admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:17.639202 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6e612-feed-46bc-8809-ea0e3ee27b6f admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:17.644296 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05045a34-f7ec-4903-a950-1c8f2cf66a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:17.644966 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05045a34-f7ec-4903-a950-1c8f2cf66a25 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:17.645240 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05045a34-f7ec-4903-a950-1c8f2cf66a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:17.645509 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05045a34-f7ec-4903-a950-1c8f2cf66a25 admin admin] PATH_INFO: `/v3/projects/4184e8c8517b4813b57ba864590cf992/users/770039941d344dc3a6ea3d7e3d379c78/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:17.664750 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05045a34-f7ec-4903-a950-1c8f2cf66a25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4184e8c8517b4813b57ba864590cf992, user_id=770039941d344dc3a6ea3d7e3d379c78, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:17.667714 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05045a34-f7ec-4903-a950-1c8f2cf66a25 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:17.684069 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1103/2203] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:46:17 2022] PUT /identity/v3/projects/4184e8c8517b4813b57ba864590cf992/users/770039941d344dc3a6ea3d7e3d379c78/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:46:17.687588 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e21f3-d37a-4687-abc1-0cc97277b8e6 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:17.700284 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e21f3-d37a-4687-abc1-0cc97277b8e6 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:17.768561 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2faf50-ad63-4a20-9bc1-92e3f1bd9f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:17.769203 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2faf50-ad63-4a20-9bc1-92e3f1bd9f16 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:17.769478 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2faf50-ad63-4a20-9bc1-92e3f1bd9f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:17.769750 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2faf50-ad63-4a20-9bc1-92e3f1bd9f16 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:17.770398 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2faf50-ad63-4a20-9bc1-92e3f1bd9f16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:17.772014 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2faf50-ad63-4a20-9bc1-92e3f1bd9f16 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:17.783894 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1101/2204] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:46:17 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:46:17.788270 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05045a34-f7ec-4903-a950-1c8f2cf66a25 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:17.799334 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05045a34-f7ec-4903-a950-1c8f2cf66a25 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:17.804828 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac1fd4-86f5-4933-b1f5-f651d6468785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:17.805632 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac1fd4-86f5-4933-b1f5-f651d6468785 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:17.805757 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac1fd4-86f5-4933-b1f5-f651d6468785 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:17.805990 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac1fd4-86f5-4933-b1f5-f651d6468785 admin admin] PATH_INFO: `/v3/projects/4184e8c8517b4813b57ba864590cf992/users/770039941d344dc3a6ea3d7e3d379c78/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:17.808789 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac1fd4-86f5-4933-b1f5-f651d6468785 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4184e8c8517b4813b57ba864590cf992, user_id=770039941d344dc3a6ea3d7e3d379c78, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:17.811737 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac1fd4-86f5-4933-b1f5-f651d6468785 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:17.854752 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1104/2205] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:46:17 2022] PUT /identity/v3/projects/4184e8c8517b4813b57ba864590cf992/users/770039941d344dc3a6ea3d7e3d379c78/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:46:18.277317 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68206d-1465-4c6c-8f5e-f16997d49bb8 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:18.277558 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68206d-1465-4c6c-8f5e-f16997d49bb8 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:18.278105 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68206d-1465-4c6c-8f5e-f16997d49bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:18.308521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-2b68206d-1465-4c6c-8f5e-f16997d49bb8 None None] MFA Rules not processed for user `c67dec924b7e4f0aa0874e6e21879889`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:46:18.310821 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-2b68206d-1465-4c6c-8f5e-f16997d49bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:18.389509 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1102/2206] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:46:18 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:46:20.490320 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac1fd4-86f5-4933-b1f5-f651d6468785 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:20.492834 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-d5ac1fd4-86f5-4933-b1f5-f651d6468785 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:20.578247 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac1fd4-86f5-4933-b1f5-f651d6468785 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:20.583551 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f638bb-175c-4cbc-982d-86414d52da96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:20.584482 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f638bb-175c-4cbc-982d-86414d52da96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:20.585014 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f638bb-175c-4cbc-982d-86414d52da96 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:20.585506 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f638bb-175c-4cbc-982d-86414d52da96 admin admin] PATH_INFO: `/v3/users/ffbf8a01a6034e7284c2f490121e0ec4` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:20.587326 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f638bb-175c-4cbc-982d-86414d52da96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffbf8a01a6034e7284c2f490121e0ec4)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:20.589805 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f638bb-175c-4cbc-982d-86414d52da96 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:20.607151 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:46:20.655371 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-e4f638bb-175c-4cbc-982d-86414d52da96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffbf8a01a6034e7284c2f490121e0ec4'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:20.656088 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-e4f638bb-175c-4cbc-982d-86414d52da96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffbf8a01a6034e7284c2f490121e0ec4'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:20.674940 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-e4f638bb-175c-4cbc-982d-86414d52da96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffbf8a01a6034e7284c2f490121e0ec4'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:20.686138 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-e4f638bb-175c-4cbc-982d-86414d52da96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffbf8a01a6034e7284c2f490121e0ec4'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:20.687086 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-e4f638bb-175c-4cbc-982d-86414d52da96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffbf8a01a6034e7284c2f490121e0ec4'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:20.706530 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1105/2207] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:46:20 2022] DELETE /identity/v3/users/ffbf8a01a6034e7284c2f490121e0ec4 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:46:20.711024 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193346f-2e54-4d6a-ae7f-63f134d39652 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:20.711488 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193346f-2e54-4d6a-ae7f-63f134d39652 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:20.711687 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193346f-2e54-4d6a-ae7f-63f134d39652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:20.751430 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-0193346f-2e54-4d6a-ae7f-63f134d39652 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:46:20.754211 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-0193346f-2e54-4d6a-ae7f-63f134d39652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:20.984490 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1103/2208] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:46:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:46:20.989498 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f638bb-175c-4cbc-982d-86414d52da96 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:20.991451 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-e4f638bb-175c-4cbc-982d-86414d52da96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:21.113354 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f638bb-175c-4cbc-982d-86414d52da96 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:21.119500 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72cd32d-0260-493b-9e82-11499940bf65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:21.121248 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72cd32d-0260-493b-9e82-11499940bf65 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:21.121731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72cd32d-0260-493b-9e82-11499940bf65 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:21.122148 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72cd32d-0260-493b-9e82-11499940bf65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:21.134673 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72cd32d-0260-493b-9e82-11499940bf65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:21.137624 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72cd32d-0260-493b-9e82-11499940bf65 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:21.160396 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1106/2209] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:46:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:46:21.655391 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894548b-1e82-4af8-967e-5956ee71e33d None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:21.655691 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894548b-1e82-4af8-967e-5956ee71e33d None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:21.655995 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894548b-1e82-4af8-967e-5956ee71e33d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:21.710043 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-6894548b-1e82-4af8-967e-5956ee71e33d None None] MFA Rules not processed for user `d0a13dbffb4a49508b7fb661cf84e803`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:46:21.721297 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-6894548b-1e82-4af8-967e-5956ee71e33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:21.721859 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72cd32d-0260-493b-9e82-11499940bf65 service neutron] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:21.723674 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-a72cd32d-0260-493b-9e82-11499940bf65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:21.741923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:46:21.758765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72cd32d-0260-493b-9e82-11499940bf65 service neutron] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:21.778927 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d564c3-d3a8-4755-8f09-6de01e1024ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:21.779915 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d564c3-d3a8-4755-8f09-6de01e1024ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:21.780330 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d564c3-d3a8-4755-8f09-6de01e1024ab admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:21.780829 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d564c3-d3a8-4755-8f09-6de01e1024ab admin admin] PATH_INFO: `/v3/projects/0b27e2dcce0b4dc4921a48dbf316825d` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:21.782170 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d564c3-d3a8-4755-8f09-6de01e1024ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b27e2dcce0b4dc4921a48dbf316825d)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:21.784577 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d564c3-d3a8-4755-8f09-6de01e1024ab admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:21.881978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-a9d564c3-d3a8-4755-8f09-6de01e1024ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b27e2dcce0b4dc4921a48dbf316825d', 'request_id': 'req-a9d564c3-d3a8-4755-8f09-6de01e1024ab'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:21.893819 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1104/2210] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Aug 17 23:46:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:46:21.920674 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-a9d564c3-d3a8-4755-8f09-6de01e1024ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b27e2dcce0b4dc4921a48dbf316825d', 'request_id': 'req-a9d564c3-d3a8-4755-8f09-6de01e1024ab'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:21.953194 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-a9d564c3-d3a8-4755-8f09-6de01e1024ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b27e2dcce0b4dc4921a48dbf316825d', 'request_id': 'req-a9d564c3-d3a8-4755-8f09-6de01e1024ab'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:22.007661 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6894548b-1e82-4af8-967e-5956ee71e33d None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:22.010960 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-6894548b-1e82-4af8-967e-5956ee71e33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:22.011535 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-a9d564c3-d3a8-4755-8f09-6de01e1024ab admin admin] The token cache is being invalidate because project 0b27e2dcce0b4dc4921a48dbf316825d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:46:22.011836 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-a9d564c3-d3a8-4755-8f09-6de01e1024ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:22.028970 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-a9d564c3-d3a8-4755-8f09-6de01e1024ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:22.034093 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1107/2211] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:46:21 2022] DELETE /identity/v3/projects/0b27e2dcce0b4dc4921a48dbf316825d => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:46:22.039575 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1035c4-26a2-49de-a188-88026ac5dc37 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:22.041160 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1035c4-26a2-49de-a188-88026ac5dc37 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:22.041836 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1035c4-26a2-49de-a188-88026ac5dc37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:22.115254 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-9f1035c4-26a2-49de-a188-88026ac5dc37 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:46:22.123991 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-9f1035c4-26a2-49de-a188-88026ac5dc37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:22.132452 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6894548b-1e82-4af8-967e-5956ee71e33d None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:22.154201 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:46:22.237204 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964e8ef-0251-4d10-bc38-c33e75b8ce3d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee422ea90557444fabdc9c292865b9b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:22.237525 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964e8ef-0251-4d10-bc38-c33e75b8ce3d service swift] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:22.237973 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964e8ef-0251-4d10-bc38-c33e75b8ce3d service swift] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:22.238197 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964e8ef-0251-4d10-bc38-c33e75b8ce3d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:22.247606 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-0964e8ef-0251-4d10-bc38-c33e75b8ce3d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:22.298179 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1108/2212] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:46:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:46:22.305553 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33143ef-1d1a-45d5-8dcc-48251bdde7c8 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:22.305842 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33143ef-1d1a-45d5-8dcc-48251bdde7c8 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:22.306188 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33143ef-1d1a-45d5-8dcc-48251bdde7c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:22.348450 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0964e8ef-0251-4d10-bc38-c33e75b8ce3d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:22.350368 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0964e8ef-0251-4d10-bc38-c33e75b8ce3d service swift] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:22.357709 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-c33143ef-1d1a-45d5-8dcc-48251bdde7c8 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:46:22.358963 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-c33143ef-1d1a-45d5-8dcc-48251bdde7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:22.360732 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:46:22.378994 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1105/2213] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Aug 17 23:46:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 17 23:46:22.409538 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1109/2214] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:46:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:46:22.413523 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964e8ef-0251-4d10-bc38-c33e75b8ce3d service swift] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:22.444584 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964e8ef-0251-4d10-bc38-c33e75b8ce3d service swift] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:22.459080 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79db61-958e-4390-8616-47935375cf04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:22.460227 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79db61-958e-4390-8616-47935375cf04 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:22.462447 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79db61-958e-4390-8616-47935375cf04 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:22.462745 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79db61-958e-4390-8616-47935375cf04 admin admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:22.464212 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa79db61-958e-4390-8616-47935375cf04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:46:22.464212 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa79db61-958e-4390-8616-47935375cf04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:22.468899 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa79db61-958e-4390-8616-47935375cf04 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:22.497700 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1106/2215] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:46:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:46:22.501999 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33143ef-1d1a-45d5-8dcc-48251bdde7c8 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:22.514124 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33143ef-1d1a-45d5-8dcc-48251bdde7c8 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:22.522322 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156ce422-1f9e-492b-a150-68eef67e4cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:22.523660 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ce422-1f9e-492b-a150-68eef67e4cb7 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:22.524280 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ce422-1f9e-492b-a150-68eef67e4cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:22.525108 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ce422-1f9e-492b-a150-68eef67e4cb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:22.526797 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156ce422-1f9e-492b-a150-68eef67e4cb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:22.529344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156ce422-1f9e-492b-a150-68eef67e4cb7 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:22.586350 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1110/2216] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:46:22 2022] POST /identity/v3/projects => generated 375 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:46:22.590229 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79db61-958e-4390-8616-47935375cf04 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:22.607232 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79db61-958e-4390-8616-47935375cf04 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:22.706049 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c12a7f-9d00-49de-bb21-276a31e38d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:22.706844 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c12a7f-9d00-49de-bb21-276a31e38d7b admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:22.707135 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c12a7f-9d00-49de-bb21-276a31e38d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:22.707416 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c12a7f-9d00-49de-bb21-276a31e38d7b admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:22.708245 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c12a7f-9d00-49de-bb21-276a31e38d7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:22.710887 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c12a7f-9d00-49de-bb21-276a31e38d7b admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:22.760476 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1107/2217] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:46:22 2022] POST /identity/v3/users => generated 288 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:46:22.761381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156ce422-1f9e-492b-a150-68eef67e4cb7 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:22.802349 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156ce422-1f9e-492b-a150-68eef67e4cb7 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:22.810705 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f2675-0d12-476b-a050-43e532ed00bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:22.811348 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f2675-0d12-476b-a050-43e532ed00bd admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:22.811634 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f2675-0d12-476b-a050-43e532ed00bd admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:22.812154 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f2675-0d12-476b-a050-43e532ed00bd admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:22.813239 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f2675-0d12-476b-a050-43e532ed00bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:22.815137 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f2675-0d12-476b-a050-43e532ed00bd admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:22.872934 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c12a7f-9d00-49de-bb21-276a31e38d7b admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:22.885953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1111/2218] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:46:22 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:46:22.889469 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c12a7f-9d00-49de-bb21-276a31e38d7b admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:22.897904 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd320a-f75f-4851-8771-183448b435ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:22.900346 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd320a-f75f-4851-8771-183448b435ac admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:22.901756 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd320a-f75f-4851-8771-183448b435ac admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:22.902425 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd320a-f75f-4851-8771-183448b435ac admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:22.921684 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebd320a-f75f-4851-8771-183448b435ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:22.928317 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebd320a-f75f-4851-8771-183448b435ac admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:22.948015 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1108/2219] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:46:22 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:46:22.950226 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f2675-0d12-476b-a050-43e532ed00bd admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:22.965092 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f2675-0d12-476b-a050-43e532ed00bd admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:22.970684 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bafaf85-3d9a-4957-9ff3-7d1ebecc456f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:22.971700 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bafaf85-3d9a-4957-9ff3-7d1ebecc456f admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:22.972306 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bafaf85-3d9a-4957-9ff3-7d1ebecc456f admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:22.973273 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bafaf85-3d9a-4957-9ff3-7d1ebecc456f admin admin] PATH_INFO: `/v3/projects/70e4add4edd046209596df51e1d045df/users/ef6c7e3700924808b310b456ff15ffa7/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:22.992530 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bafaf85-3d9a-4957-9ff3-7d1ebecc456f admin admin] RBAC: Authorizing `identity:create_grant(project_id=70e4add4edd046209596df51e1d045df, user_id=ef6c7e3700924808b310b456ff15ffa7, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:22.995627 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bafaf85-3d9a-4957-9ff3-7d1ebecc456f admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:23.011605 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1112/2220] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:46:22 2022] PUT /identity/v3/projects/70e4add4edd046209596df51e1d045df/users/ef6c7e3700924808b310b456ff15ffa7/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:46:23.022332 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c25026-1913-4bc1-87eb-804b6dad290e None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:23.022625 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c25026-1913-4bc1-87eb-804b6dad290e None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:23.022905 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c25026-1913-4bc1-87eb-804b6dad290e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:23.069119 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-e8c25026-1913-4bc1-87eb-804b6dad290e None None] MFA Rules not processed for user `ef6c7e3700924808b310b456ff15ffa7`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:46:23.070272 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-e8c25026-1913-4bc1-87eb-804b6dad290e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:23.182801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1109/2221] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:46:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:46:23.189503 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bafaf85-3d9a-4957-9ff3-7d1ebecc456f admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:23.191227 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-4bafaf85-3d9a-4957-9ff3-7d1ebecc456f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:23.276547 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f907c08-e8fb-45c5-a8b5-209a62a86dbf None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:23.277221 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f907c08-e8fb-45c5-a8b5-209a62a86dbf None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:23.277732 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f907c08-e8fb-45c5-a8b5-209a62a86dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:23.305406 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bafaf85-3d9a-4957-9ff3-7d1ebecc456f admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:23.309011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-2f907c08-e8fb-45c5-a8b5-209a62a86dbf None None] MFA Rules not processed for user `d0a13dbffb4a49508b7fb661cf84e803`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:46:23.311604 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-2f907c08-e8fb-45c5-a8b5-209a62a86dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:23.316385 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b2b009-9e15-4ba1-9416-4d4b0aac59ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:23.317358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2b009-9e15-4ba1-9416-4d4b0aac59ed service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:23.318664 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2b009-9e15-4ba1-9416-4d4b0aac59ed service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:23.319405 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2b009-9e15-4ba1-9416-4d4b0aac59ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:23.334129 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b2b009-9e15-4ba1-9416-4d4b0aac59ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:23.335825 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b2b009-9e15-4ba1-9416-4d4b0aac59ed service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:23.348330 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1113/2222] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:46:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:46:23.394415 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:46:23.443091 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1110/2223] 10.222.0.26 () {58 vars in 967 bytes} [Wed Aug 17 23:46:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:46:23.476005 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b2b009-9e15-4ba1-9416-4d4b0aac59ed service cinder] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:23.477684 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-94b2b009-9e15-4ba1-9416-4d4b0aac59ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:23.602291 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:46:23.636716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b2b009-9e15-4ba1-9416-4d4b0aac59ed service cinder] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:23.644453 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e0772-5254-49f3-a015-65c67db23898 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee422ea90557444fabdc9c292865b9b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:23.649285 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e0772-5254-49f3-a015-65c67db23898 service swift] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:23.649968 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e0772-5254-49f3-a015-65c67db23898 service swift] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:23.650306 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e0772-5254-49f3-a015-65c67db23898 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:23.660533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e0772-5254-49f3-a015-65c67db23898 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:23.662701 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e0772-5254-49f3-a015-65c67db23898 service swift] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:23.680651 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1114/2224] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Aug 17 23:46:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 23:46:24.226748 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21829ae6-b1e9-4b51-90c0-cf15cd3d9fac None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:24.227719 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21829ae6-b1e9-4b51-90c0-cf15cd3d9fac None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:24.228239 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21829ae6-b1e9-4b51-90c0-cf15cd3d9fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:24.249758 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-21829ae6-b1e9-4b51-90c0-cf15cd3d9fac None None] MFA Rules not processed for user `d0a13dbffb4a49508b7fb661cf84e803`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:46:24.251429 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-21829ae6-b1e9-4b51-90c0-cf15cd3d9fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:24.259817 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1111/2225] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Aug 17 23:46:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:46:24.273908 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e0772-5254-49f3-a015-65c67db23898 service swift] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:24.287229 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e0772-5254-49f3-a015-65c67db23898 service swift] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:24.312778 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668bc413-2df8-41f4-9672-2615dbb2b90e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee422ea90557444fabdc9c292865b9b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:24.314038 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bc413-2df8-41f4-9672-2615dbb2b90e service swift] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:24.314515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bc413-2df8-41f4-9672-2615dbb2b90e service swift] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:24.315044 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bc413-2df8-41f4-9672-2615dbb2b90e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:24.328125 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668bc413-2df8-41f4-9672-2615dbb2b90e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:24.330074 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668bc413-2df8-41f4-9672-2615dbb2b90e service swift] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:24.349467 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1115/2226] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Aug 17 23:46:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 17 23:46:24.924349 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da9043a-93a1-44be-b53e-425cf9b7ccd2 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:24.924922 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da9043a-93a1-44be-b53e-425cf9b7ccd2 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:24.925358 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da9043a-93a1-44be-b53e-425cf9b7ccd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:24.969026 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-6da9043a-93a1-44be-b53e-425cf9b7ccd2 None None] MFA Rules not processed for user `d0a13dbffb4a49508b7fb661cf84e803`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:46:24.970253 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-6da9043a-93a1-44be-b53e-425cf9b7ccd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:24.979654 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1112/2227] 10.222.0.26 () {58 vars in 967 bytes} [Wed Aug 17 23:46:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:46:24.995178 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668bc413-2df8-41f4-9672-2615dbb2b90e service swift] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:25.007896 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668bc413-2df8-41f4-9672-2615dbb2b90e service swift] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:25.016477 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453c8ef-b38b-45ae-ad39-6c8936227fe0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee422ea90557444fabdc9c292865b9b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:25.017213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453c8ef-b38b-45ae-ad39-6c8936227fe0 service swift] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:25.017589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453c8ef-b38b-45ae-ad39-6c8936227fe0 service swift] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:25.017956 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453c8ef-b38b-45ae-ad39-6c8936227fe0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:25.034261 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453c8ef-b38b-45ae-ad39-6c8936227fe0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:25.036401 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453c8ef-b38b-45ae-ad39-6c8936227fe0 service swift] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:25.047158 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1116/2228] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Aug 17 23:46:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 23:46:25.309858 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe6663-821f-4969-9dd7-4bda228084f8 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:25.310917 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe6663-821f-4969-9dd7-4bda228084f8 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:25.311411 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe6663-821f-4969-9dd7-4bda228084f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:25.366885 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-dbbe6663-821f-4969-9dd7-4bda228084f8 None None] MFA Rules not processed for user `770039941d344dc3a6ea3d7e3d379c78`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:46:25.368629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-dbbe6663-821f-4969-9dd7-4bda228084f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:25.568538 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1113/2229] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:46:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:46:25.591107 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453c8ef-b38b-45ae-ad39-6c8936227fe0 service swift] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:25.593107 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-a453c8ef-b38b-45ae-ad39-6c8936227fe0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:25.756576 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453c8ef-b38b-45ae-ad39-6c8936227fe0 service swift] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:25.768750 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234f8688-f43f-46e2-be35-9566d1c6ee36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:25.769999 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f8688-f43f-46e2-be35-9566d1c6ee36 service nova] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:25.770430 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f8688-f43f-46e2-be35-9566d1c6ee36 service nova] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:25.770816 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f8688-f43f-46e2-be35-9566d1c6ee36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:25.774674 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-234f8688-f43f-46e2-be35-9566d1c6ee36 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:25.893896 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234f8688-f43f-46e2-be35-9566d1c6ee36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:25.911874 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234f8688-f43f-46e2-be35-9566d1c6ee36 service nova] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:25.935841 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1117/2230] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:46:25 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:46:27.508677 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe6663-821f-4969-9dd7-4bda228084f8 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:27.510706 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-dbbe6663-821f-4969-9dd7-4bda228084f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:27.582910 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe6663-821f-4969-9dd7-4bda228084f8 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:27.587595 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a73129-e1f6-4eae-9f5e-eec0569211dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:27.588369 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a73129-e1f6-4eae-9f5e-eec0569211dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:27.588717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a73129-e1f6-4eae-9f5e-eec0569211dc admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:27.589060 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a73129-e1f6-4eae-9f5e-eec0569211dc admin admin] PATH_INFO: `/v3/users/f34ca182eb1c4627beb18ac1256607f8` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:27.590265 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a73129-e1f6-4eae-9f5e-eec0569211dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f34ca182eb1c4627beb18ac1256607f8)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:27.592304 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a73129-e1f6-4eae-9f5e-eec0569211dc admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:27.647011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-20a73129-e1f6-4eae-9f5e-eec0569211dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f34ca182eb1c4627beb18ac1256607f8'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:27.647688 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-20a73129-e1f6-4eae-9f5e-eec0569211dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f34ca182eb1c4627beb18ac1256607f8'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:27.648927 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-20a73129-e1f6-4eae-9f5e-eec0569211dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f34ca182eb1c4627beb18ac1256607f8'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:27.657388 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-20a73129-e1f6-4eae-9f5e-eec0569211dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f34ca182eb1c4627beb18ac1256607f8'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:27.668301 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-20a73129-e1f6-4eae-9f5e-eec0569211dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f34ca182eb1c4627beb18ac1256607f8'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:27.677629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:46:27.689917 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1114/2231] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:46:27 2022] DELETE /identity/v3/users/f34ca182eb1c4627beb18ac1256607f8 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:46:27.694815 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f62dc6-8f5b-43df-b039-bf18a5d80ecd None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:27.695327 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f62dc6-8f5b-43df-b039-bf18a5d80ecd None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:27.695442 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f62dc6-8f5b-43df-b039-bf18a5d80ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:27.715315 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-c3f62dc6-8f5b-43df-b039-bf18a5d80ecd None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:46:27.716796 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-c3f62dc6-8f5b-43df-b039-bf18a5d80ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:27.782499 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:46:27.805370 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1118/2232] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:46:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:46:27.814280 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a73129-e1f6-4eae-9f5e-eec0569211dc admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:27.816197 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-20a73129-e1f6-4eae-9f5e-eec0569211dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:27.914106 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a73129-e1f6-4eae-9f5e-eec0569211dc admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:27.917988 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649bcda0-91b0-45ac-9b0c-ee8748ef3984 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:27.919322 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649bcda0-91b0-45ac-9b0c-ee8748ef3984 service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:27.919697 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649bcda0-91b0-45ac-9b0c-ee8748ef3984 service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:27.920120 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649bcda0-91b0-45ac-9b0c-ee8748ef3984 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:27.933295 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649bcda0-91b0-45ac-9b0c-ee8748ef3984 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:27.934731 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649bcda0-91b0-45ac-9b0c-ee8748ef3984 service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:27.946375 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1115/2233] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:46:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:46:28.168942 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f62dc6-8f5b-43df-b039-bf18a5d80ecd None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:28.170392 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-c3f62dc6-8f5b-43df-b039-bf18a5d80ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:28.182622 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f62dc6-8f5b-43df-b039-bf18a5d80ecd None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:28.188603 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a3b7a-ffcb-4ba7-8423-5e009a44646f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:28.189252 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a3b7a-ffcb-4ba7-8423-5e009a44646f admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:28.189528 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a3b7a-ffcb-4ba7-8423-5e009a44646f admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:28.190037 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a3b7a-ffcb-4ba7-8423-5e009a44646f admin admin] PATH_INFO: `/v3/projects/9f504189f5a949cdae85a009e3b182b1` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:28.191771 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71a3b7a-ffcb-4ba7-8423-5e009a44646f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f504189f5a949cdae85a009e3b182b1)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:28.194443 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71a3b7a-ffcb-4ba7-8423-5e009a44646f admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:28.318331 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-f71a3b7a-ffcb-4ba7-8423-5e009a44646f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f504189f5a949cdae85a009e3b182b1', 'request_id': 'req-f71a3b7a-ffcb-4ba7-8423-5e009a44646f'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:28.327404 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-f71a3b7a-ffcb-4ba7-8423-5e009a44646f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f504189f5a949cdae85a009e3b182b1', 'request_id': 'req-f71a3b7a-ffcb-4ba7-8423-5e009a44646f'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:28.344868 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-f71a3b7a-ffcb-4ba7-8423-5e009a44646f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f504189f5a949cdae85a009e3b182b1', 'request_id': 'req-f71a3b7a-ffcb-4ba7-8423-5e009a44646f'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:28.364294 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-f71a3b7a-ffcb-4ba7-8423-5e009a44646f admin admin] The token cache is being invalidate because project 9f504189f5a949cdae85a009e3b182b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:46:28.364605 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-f71a3b7a-ffcb-4ba7-8423-5e009a44646f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:28.365620 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-f71a3b7a-ffcb-4ba7-8423-5e009a44646f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:28.371211 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1119/2234] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:46:28 2022] DELETE /identity/v3/projects/9f504189f5a949cdae85a009e3b182b1 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:46:28.372853 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4714ad-e79d-4da1-8bc4-7b0bd3dcd01c None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:28.373150 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4714ad-e79d-4da1-8bc4-7b0bd3dcd01c None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:28.373434 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4714ad-e79d-4da1-8bc4-7b0bd3dcd01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:28.394201 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-8d4714ad-e79d-4da1-8bc4-7b0bd3dcd01c None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:46:28.395448 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-8d4714ad-e79d-4da1-8bc4-7b0bd3dcd01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:28.479856 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1116/2235] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:46:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:46:28.491261 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcaf073-dd03-4325-bb3d-a774e4b00b07 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:28.491536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcaf073-dd03-4325-bb3d-a774e4b00b07 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:28.491827 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcaf073-dd03-4325-bb3d-a774e4b00b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:28.512914 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-ddcaf073-dd03-4325-bb3d-a774e4b00b07 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:46:28.514124 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-ddcaf073-dd03-4325-bb3d-a774e4b00b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:28.522296 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1120/2236] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:46:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:46:28.526577 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4714ad-e79d-4da1-8bc4-7b0bd3dcd01c None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:28.536958 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4714ad-e79d-4da1-8bc4-7b0bd3dcd01c None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:28.543841 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20885048-b9dd-4ec6-bed4-f9d9fae70224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:28.544494 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20885048-b9dd-4ec6-bed4-f9d9fae70224 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:28.544779 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20885048-b9dd-4ec6-bed4-f9d9fae70224 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:28.545053 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20885048-b9dd-4ec6-bed4-f9d9fae70224 admin admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:28.545715 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20885048-b9dd-4ec6-bed4-f9d9fae70224 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:46:28.546020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20885048-b9dd-4ec6-bed4-f9d9fae70224 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:28.547684 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20885048-b9dd-4ec6-bed4-f9d9fae70224 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:28.561056 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1117/2237] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:46:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:46:28.565259 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcaf073-dd03-4325-bb3d-a774e4b00b07 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:28.574736 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcaf073-dd03-4325-bb3d-a774e4b00b07 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:28.579679 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b47e67f-9ed1-47d9-abf2-6cbd05913dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:28.580333 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47e67f-9ed1-47d9-abf2-6cbd05913dee admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:28.580611 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47e67f-9ed1-47d9-abf2-6cbd05913dee admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:28.580880 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47e67f-9ed1-47d9-abf2-6cbd05913dee admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:28.581526 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b47e67f-9ed1-47d9-abf2-6cbd05913dee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:28.583448 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b47e67f-9ed1-47d9-abf2-6cbd05913dee admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:28.612610 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1121/2238] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:46:28 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:46:28.616863 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20885048-b9dd-4ec6-bed4-f9d9fae70224 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:28.628545 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20885048-b9dd-4ec6-bed4-f9d9fae70224 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:28.690500 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bffd4e-8566-4dc6-a2c1-8936258b5578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:28.691461 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bffd4e-8566-4dc6-a2c1-8936258b5578 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:28.691920 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bffd4e-8566-4dc6-a2c1-8936258b5578 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:28.692348 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bffd4e-8566-4dc6-a2c1-8936258b5578 admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:28.693140 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bffd4e-8566-4dc6-a2c1-8936258b5578 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:28.695419 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bffd4e-8566-4dc6-a2c1-8936258b5578 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:28.719581 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1118/2239] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:46:28 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:46:28.723753 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b47e67f-9ed1-47d9-abf2-6cbd05913dee admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:28.745030 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b47e67f-9ed1-47d9-abf2-6cbd05913dee admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:28.752456 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b3460-a4b0-401e-8aa1-2564036f67dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:28.753198 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b3460-a4b0-401e-8aa1-2564036f67dc admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:28.753754 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b3460-a4b0-401e-8aa1-2564036f67dc admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:28.754123 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b3460-a4b0-401e-8aa1-2564036f67dc admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:28.754897 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28b3460-a4b0-401e-8aa1-2564036f67dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:28.756660 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28b3460-a4b0-401e-8aa1-2564036f67dc admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:28.776085 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1122/2240] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 23:46:28.780344 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bffd4e-8566-4dc6-a2c1-8936258b5578 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:28.792508 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bffd4e-8566-4dc6-a2c1-8936258b5578 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:28.797556 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c89812-6bab-4d17-84e2-f14a8842a7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:28.798390 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c89812-6bab-4d17-84e2-f14a8842a7a9 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:28.798743 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c89812-6bab-4d17-84e2-f14a8842a7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:28.799096 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c89812-6bab-4d17-84e2-f14a8842a7a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:28.799821 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c89812-6bab-4d17-84e2-f14a8842a7a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:28.801651 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c89812-6bab-4d17-84e2-f14a8842a7a9 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:28.808917 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:46:28.827845 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1119/2241] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:46:28.832085 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b3460-a4b0-401e-8aa1-2564036f67dc admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:28.842564 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b3460-a4b0-401e-8aa1-2564036f67dc admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:28.847558 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a0468-3fc4-4934-ace1-2f798e3989d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:28.848462 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a0468-3fc4-4934-ace1-2f798e3989d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:28.848739 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a0468-3fc4-4934-ace1-2f798e3989d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:28.849011 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a0468-3fc4-4934-ace1-2f798e3989d8 admin admin] PATH_INFO: `/v3/projects/aac208652b8247049fa6a0bc9aa991bf/users/ce1b5129d46345ef99c69e95f9824c45/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:28.873140 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a0468-3fc4-4934-ace1-2f798e3989d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aac208652b8247049fa6a0bc9aa991bf, user_id=ce1b5129d46345ef99c69e95f9824c45, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:28.876175 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a0468-3fc4-4934-ace1-2f798e3989d8 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:28.893392 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1123/2242] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:46:28 2022] PUT /identity/v3/projects/aac208652b8247049fa6a0bc9aa991bf/users/ce1b5129d46345ef99c69e95f9824c45/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:46:28.898859 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dfd09b-5bf2-482e-8115-857ac100401f None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:28.899148 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dfd09b-5bf2-482e-8115-857ac100401f None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:28.899427 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dfd09b-5bf2-482e-8115-857ac100401f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:28.921350 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-99dfd09b-5bf2-482e-8115-857ac100401f None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:46:28.922961 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-99dfd09b-5bf2-482e-8115-857ac100401f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:29.012756 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1120/2243] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:46:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:46:29.023085 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a0468-3fc4-4934-ace1-2f798e3989d8 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:29.025273 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-bc7a0468-3fc4-4934-ace1-2f798e3989d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:29.106120 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a0468-3fc4-4934-ace1-2f798e3989d8 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:29.110793 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb2b9f4-c9f1-4520-afe4-d0a9c154084f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:29.111335 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb2b9f4-c9f1-4520-afe4-d0a9c154084f service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:29.111624 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb2b9f4-c9f1-4520-afe4-d0a9c154084f service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:29.111906 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb2b9f4-c9f1-4520-afe4-d0a9c154084f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:29.123428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb2b9f4-c9f1-4520-afe4-d0a9c154084f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:29.125004 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb2b9f4-c9f1-4520-afe4-d0a9c154084f service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:29.135790 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1124/2244] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:46:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 23:46:34.777023 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1c3d3-fe14-4c4f-a3ac-22fe0a4bc292 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:34.777581 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1c3d3-fe14-4c4f-a3ac-22fe0a4bc292 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:34.777581 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1c3d3-fe14-4c4f-a3ac-22fe0a4bc292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:34.811202 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-a9a1c3d3-fe14-4c4f-a3ac-22fe0a4bc292 None None] MFA Rules not processed for user `ce1b5129d46345ef99c69e95f9824c45`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:46:34.812793 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-a9a1c3d3-fe14-4c4f-a3ac-22fe0a4bc292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:34.900032 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1121/2245] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:46:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:46:34.905211 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb2b9f4-c9f1-4520-afe4-d0a9c154084f service neutron] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:34.946033 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb2b9f4-c9f1-4520-afe4-d0a9c154084f service neutron] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:34.975605 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dc933e-1035-4d43-9760-6e7e8c234674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:34.976397 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc933e-1035-4d43-9760-6e7e8c234674 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:34.976673 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc933e-1035-4d43-9760-6e7e8c234674 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:34.976944 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc933e-1035-4d43-9760-6e7e8c234674 admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:34.977589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dc933e-1035-4d43-9760-6e7e8c234674 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:34.984516 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dc933e-1035-4d43-9760-6e7e8c234674 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:35.035347 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:46:35.052059 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a1c3d3-fe14-4c4f-a3ac-22fe0a4bc292 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:35.058758 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1125/2246] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:46:34 2022] POST /identity/v3/projects => generated 367 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:46:35.069311 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a1c3d3-fe14-4c4f-a3ac-22fe0a4bc292 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:35.163127 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a824f-85c0-4342-b78f-11e272eb0e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:35.163886 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a824f-85c0-4342-b78f-11e272eb0e0c admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:35.164209 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a824f-85c0-4342-b78f-11e272eb0e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:35.164487 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a824f-85c0-4342-b78f-11e272eb0e0c admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:35.165128 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56a824f-85c0-4342-b78f-11e272eb0e0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:35.167639 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56a824f-85c0-4342-b78f-11e272eb0e0c admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:35.191430 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1122/2247] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:46:35 2022] POST /identity/v3/users => generated 290 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:46:35.195427 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dc933e-1035-4d43-9760-6e7e8c234674 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:35.206231 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dc933e-1035-4d43-9760-6e7e8c234674 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:35.210987 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ebde2-b806-497f-a78b-fb28b99314df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:35.211451 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ebde2-b806-497f-a78b-fb28b99314df admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:35.211725 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ebde2-b806-497f-a78b-fb28b99314df admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:35.212009 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ebde2-b806-497f-a78b-fb28b99314df admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:35.212659 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ebde2-b806-497f-a78b-fb28b99314df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:35.214819 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ebde2-b806-497f-a78b-fb28b99314df admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:35.226179 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1126/2248] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:46:35.231225 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a824f-85c0-4342-b78f-11e272eb0e0c admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:35.241812 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a824f-85c0-4342-b78f-11e272eb0e0c admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:35.247518 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e855fd3-04eb-4fd2-8313-434e7f2d9a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:35.248241 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e855fd3-04eb-4fd2-8313-434e7f2d9a0f admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:35.248539 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e855fd3-04eb-4fd2-8313-434e7f2d9a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:35.248812 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e855fd3-04eb-4fd2-8313-434e7f2d9a0f admin admin] PATH_INFO: `/v3/projects/25139a07847c44f89448109ec8fadd56/users/963f8c6c9f6d4b1aa076802c21021ae8/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:35.267118 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e855fd3-04eb-4fd2-8313-434e7f2d9a0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=25139a07847c44f89448109ec8fadd56, user_id=963f8c6c9f6d4b1aa076802c21021ae8, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:35.270082 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e855fd3-04eb-4fd2-8313-434e7f2d9a0f admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:35.274581 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:46:35.291922 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1123/2249] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:46:35 2022] PUT /identity/v3/projects/25139a07847c44f89448109ec8fadd56/users/963f8c6c9f6d4b1aa076802c21021ae8/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:46:35.293259 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ebde2-b806-497f-a78b-fb28b99314df admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:35.304551 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ebde2-b806-497f-a78b-fb28b99314df admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:35.368540 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114bf023-8aa0-4696-8e73-269f1f06ab63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:35.369372 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bf023-8aa0-4696-8e73-269f1f06ab63 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:35.369706 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bf023-8aa0-4696-8e73-269f1f06ab63 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:35.369987 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bf023-8aa0-4696-8e73-269f1f06ab63 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:35.370696 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114bf023-8aa0-4696-8e73-269f1f06ab63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:35.372549 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114bf023-8aa0-4696-8e73-269f1f06ab63 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:35.389822 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1127/2250] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:46:35.391377 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e855fd3-04eb-4fd2-8313-434e7f2d9a0f admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:35.403064 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e855fd3-04eb-4fd2-8313-434e7f2d9a0f admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:35.408575 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8ba37-f5cb-4b67-ba97-738d2ea350ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:35.409356 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8ba37-f5cb-4b67-ba97-738d2ea350ca admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:35.409656 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8ba37-f5cb-4b67-ba97-738d2ea350ca admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:35.410320 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8ba37-f5cb-4b67-ba97-738d2ea350ca admin admin] PATH_INFO: `/v3/projects/25139a07847c44f89448109ec8fadd56/users/963f8c6c9f6d4b1aa076802c21021ae8/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:35.412776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8ba37-f5cb-4b67-ba97-738d2ea350ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=25139a07847c44f89448109ec8fadd56, user_id=963f8c6c9f6d4b1aa076802c21021ae8, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:35.415189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8ba37-f5cb-4b67-ba97-738d2ea350ca admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:35.426988 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1124/2251] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:46:35 2022] PUT /identity/v3/projects/25139a07847c44f89448109ec8fadd56/users/963f8c6c9f6d4b1aa076802c21021ae8/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:46:36.158458 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114bf023-8aa0-4696-8e73-269f1f06ab63 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:36.165433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-114bf023-8aa0-4696-8e73-269f1f06ab63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:36.180335 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114bf023-8aa0-4696-8e73-269f1f06ab63 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:36.288294 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5935eb0-0694-4cfb-a7f7-99823596c496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:36.288940 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5935eb0-0694-4cfb-a7f7-99823596c496 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:36.289361 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5935eb0-0694-4cfb-a7f7-99823596c496 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:36.289642 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5935eb0-0694-4cfb-a7f7-99823596c496 admin admin] PATH_INFO: `/v3/users/ef6c7e3700924808b310b456ff15ffa7` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:36.290802 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5935eb0-0694-4cfb-a7f7-99823596c496 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef6c7e3700924808b310b456ff15ffa7)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:36.293390 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5935eb0-0694-4cfb-a7f7-99823596c496 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:36.371938 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-b5935eb0-0694-4cfb-a7f7-99823596c496 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef6c7e3700924808b310b456ff15ffa7'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:36.372674 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-b5935eb0-0694-4cfb-a7f7-99823596c496 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef6c7e3700924808b310b456ff15ffa7'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:36.396709 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-b5935eb0-0694-4cfb-a7f7-99823596c496 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef6c7e3700924808b310b456ff15ffa7'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:36.422898 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-b5935eb0-0694-4cfb-a7f7-99823596c496 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef6c7e3700924808b310b456ff15ffa7'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:36.425082 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-b5935eb0-0694-4cfb-a7f7-99823596c496 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef6c7e3700924808b310b456ff15ffa7'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:36.459589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1128/2252] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:46:36 2022] DELETE /identity/v3/users/ef6c7e3700924808b310b456ff15ffa7 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:46:36.461995 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b650f4-05d1-40c8-b75e-fb7acd7faca4 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:36.462814 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b650f4-05d1-40c8-b75e-fb7acd7faca4 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:36.464087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b650f4-05d1-40c8-b75e-fb7acd7faca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:36.496225 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-f4b650f4-05d1-40c8-b75e-fb7acd7faca4 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:46:36.497678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-f4b650f4-05d1-40c8-b75e-fb7acd7faca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:36.566748 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1125/2253] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:46:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:46:36.578234 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5935eb0-0694-4cfb-a7f7-99823596c496 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:36.579806 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-b5935eb0-0694-4cfb-a7f7-99823596c496 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:36.585523 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:46:36.682877 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5935eb0-0694-4cfb-a7f7-99823596c496 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:36.690102 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a129b9-16ba-444f-a597-367e3acb9832 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:36.690509 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a129b9-16ba-444f-a597-367e3acb9832 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:36.690785 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a129b9-16ba-444f-a597-367e3acb9832 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:36.691060 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a129b9-16ba-444f-a597-367e3acb9832 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:36.708841 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a129b9-16ba-444f-a597-367e3acb9832 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:36.710490 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a129b9-16ba-444f-a597-367e3acb9832 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:36.739262 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1129/2254] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:46:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:46:37.405086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b650f4-05d1-40c8-b75e-fb7acd7faca4 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:37.406885 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-f4b650f4-05d1-40c8-b75e-fb7acd7faca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:37.430838 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b650f4-05d1-40c8-b75e-fb7acd7faca4 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:37.445841 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdfc0d6-1604-4f11-aaca-d292e12f16b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:37.446921 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdfc0d6-1604-4f11-aaca-d292e12f16b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:37.450946 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdfc0d6-1604-4f11-aaca-d292e12f16b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:37.451817 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdfc0d6-1604-4f11-aaca-d292e12f16b5 admin admin] PATH_INFO: `/v3/projects/70e4add4edd046209596df51e1d045df` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:37.456307 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdfc0d6-1604-4f11-aaca-d292e12f16b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70e4add4edd046209596df51e1d045df)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:37.461821 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdfc0d6-1604-4f11-aaca-d292e12f16b5 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:37.570976 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-acdfc0d6-1604-4f11-aaca-d292e12f16b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70e4add4edd046209596df51e1d045df', 'request_id': 'req-acdfc0d6-1604-4f11-aaca-d292e12f16b5'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:37.580543 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-acdfc0d6-1604-4f11-aaca-d292e12f16b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70e4add4edd046209596df51e1d045df', 'request_id': 'req-acdfc0d6-1604-4f11-aaca-d292e12f16b5'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:37.590242 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-acdfc0d6-1604-4f11-aaca-d292e12f16b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70e4add4edd046209596df51e1d045df', 'request_id': 'req-acdfc0d6-1604-4f11-aaca-d292e12f16b5'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:37.605691 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-acdfc0d6-1604-4f11-aaca-d292e12f16b5 admin admin] The token cache is being invalidate because project 70e4add4edd046209596df51e1d045df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:46:37.606154 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-acdfc0d6-1604-4f11-aaca-d292e12f16b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:37.606917 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-acdfc0d6-1604-4f11-aaca-d292e12f16b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:46:37.615682 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1126/2255] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:46:37 2022] DELETE /identity/v3/projects/70e4add4edd046209596df51e1d045df => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:46:37.617692 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d59852-ce63-45e7-9e44-ed45f6f4f824 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:37.618158 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d59852-ce63-45e7-9e44-ed45f6f4f824 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:37.618288 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d59852-ce63-45e7-9e44-ed45f6f4f824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:37.648821 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-30d59852-ce63-45e7-9e44-ed45f6f4f824 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:46:37.650771 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-30d59852-ce63-45e7-9e44-ed45f6f4f824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:37.731354 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1130/2256] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:46:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:46:37.802330 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff59c8-7aaa-431b-8740-b31b1a59ce8d None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:37.803523 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff59c8-7aaa-431b-8740-b31b1a59ce8d None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:37.803930 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff59c8-7aaa-431b-8740-b31b1a59ce8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:37.843050 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-daff59c8-7aaa-431b-8740-b31b1a59ce8d None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:46:37.845025 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-daff59c8-7aaa-431b-8740-b31b1a59ce8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:37.859944 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1127/2257] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:46:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:46:37.864335 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d59852-ce63-45e7-9e44-ed45f6f4f824 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:37.876332 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d59852-ce63-45e7-9e44-ed45f6f4f824 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:37.883917 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444fec2d-7b86-44e6-a8aa-35d7492ddf97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:37.884776 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444fec2d-7b86-44e6-a8aa-35d7492ddf97 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:37.885159 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444fec2d-7b86-44e6-a8aa-35d7492ddf97 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:37.885534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444fec2d-7b86-44e6-a8aa-35d7492ddf97 admin admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:37.886307 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444fec2d-7b86-44e6-a8aa-35d7492ddf97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:46:37.886733 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444fec2d-7b86-44e6-a8aa-35d7492ddf97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:37.888832 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444fec2d-7b86-44e6-a8aa-35d7492ddf97 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:37.915156 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1131/2258] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:46:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:46:37.917031 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daff59c8-7aaa-431b-8740-b31b1a59ce8d None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:37.928503 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daff59c8-7aaa-431b-8740-b31b1a59ce8d None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:37.941899 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a0e903-7527-4f1f-812a-e8eb0c68894d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:37.942800 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0e903-7527-4f1f-812a-e8eb0c68894d admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:37.943233 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0e903-7527-4f1f-812a-e8eb0c68894d admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:37.943654 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0e903-7527-4f1f-812a-e8eb0c68894d admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:37.944561 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a0e903-7527-4f1f-812a-e8eb0c68894d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:37.948600 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a0e903-7527-4f1f-812a-e8eb0c68894d admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:38.009778 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1128/2259] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:46:37 2022] POST /identity/v3/projects => generated 385 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:46:38.013970 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444fec2d-7b86-44e6-a8aa-35d7492ddf97 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:38.026598 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444fec2d-7b86-44e6-a8aa-35d7492ddf97 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:38.124896 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4437b0-fcef-4fc1-9d7f-1cccbb2335b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:38.125534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4437b0-fcef-4fc1-9d7f-1cccbb2335b3 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:38.125809 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4437b0-fcef-4fc1-9d7f-1cccbb2335b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:38.126079 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4437b0-fcef-4fc1-9d7f-1cccbb2335b3 admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:38.126723 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4437b0-fcef-4fc1-9d7f-1cccbb2335b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:38.128778 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4437b0-fcef-4fc1-9d7f-1cccbb2335b3 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:38.159908 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1132/2260] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:46:38 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:46:38.163882 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a0e903-7527-4f1f-812a-e8eb0c68894d admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:38.185152 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a0e903-7527-4f1f-812a-e8eb0c68894d admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:38.199233 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f9e425-af06-44a5-af97-17e3b4167f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:38.200075 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9e425-af06-44a5-af97-17e3b4167f4d admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:38.200479 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9e425-af06-44a5-af97-17e3b4167f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:38.200876 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9e425-af06-44a5-af97-17e3b4167f4d admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:38.201663 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f9e425-af06-44a5-af97-17e3b4167f4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:38.203553 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f9e425-af06-44a5-af97-17e3b4167f4d admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:38.230415 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1129/2261] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:46:38 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:46:38.234589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4437b0-fcef-4fc1-9d7f-1cccbb2335b3 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:38.260792 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4437b0-fcef-4fc1-9d7f-1cccbb2335b3 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:38.275439 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3492b596-3f17-4d1f-8921-601556d8d191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:38.276860 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492b596-3f17-4d1f-8921-601556d8d191 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:38.277083 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492b596-3f17-4d1f-8921-601556d8d191 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:38.277362 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492b596-3f17-4d1f-8921-601556d8d191 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:38.278917 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3492b596-3f17-4d1f-8921-601556d8d191 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:38.280611 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3492b596-3f17-4d1f-8921-601556d8d191 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:38.302901 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1133/2262] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:46:38 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:46:38.307131 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f9e425-af06-44a5-af97-17e3b4167f4d admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:38.318512 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f9e425-af06-44a5-af97-17e3b4167f4d admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:38.330555 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4570f03e-e5ee-42fe-bd7b-d0e62e919b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:38.331231 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570f03e-e5ee-42fe-bd7b-d0e62e919b3a admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:38.331521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570f03e-e5ee-42fe-bd7b-d0e62e919b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:38.331800 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570f03e-e5ee-42fe-bd7b-d0e62e919b3a admin admin] PATH_INFO: `/v3/projects/b3ff5d4d64a740cabdc0144fe0b475ea/users/006f31b802f94549a517df90180dca36/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:38.361890 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4570f03e-e5ee-42fe-bd7b-d0e62e919b3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3ff5d4d64a740cabdc0144fe0b475ea, user_id=006f31b802f94549a517df90180dca36, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:38.365179 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4570f03e-e5ee-42fe-bd7b-d0e62e919b3a admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:38.398414 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1130/2263] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:46:38 2022] PUT /identity/v3/projects/b3ff5d4d64a740cabdc0144fe0b475ea/users/006f31b802f94549a517df90180dca36/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:46:38.404927 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1a7a9-430a-4562-8b51-dda695df9852 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:38.404927 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1a7a9-430a-4562-8b51-dda695df9852 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:38.404927 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1a7a9-430a-4562-8b51-dda695df9852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:38.449538 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-0fa1a7a9-430a-4562-8b51-dda695df9852 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:46:38.454549 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-0fa1a7a9-430a-4562-8b51-dda695df9852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:38.478726 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:46:38.590631 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1134/2264] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:46:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:46:38.613923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4570f03e-e5ee-42fe-bd7b-d0e62e919b3a admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:38.616477 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-4570f03e-e5ee-42fe-bd7b-d0e62e919b3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:38.623548 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:46:38.818206 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4570f03e-e5ee-42fe-bd7b-d0e62e919b3a admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:38.840929 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d9162-5753-48af-ba30-13e7d0090f91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:38.842139 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d9162-5753-48af-ba30-13e7d0090f91 service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:38.842711 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d9162-5753-48af-ba30-13e7d0090f91 service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:38.843153 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d9162-5753-48af-ba30-13e7d0090f91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:38.875224 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504d9162-5753-48af-ba30-13e7d0090f91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:38.877972 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504d9162-5753-48af-ba30-13e7d0090f91 service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:38.903218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1131/2265] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:46:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:46:41.560677 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a2db9-77c2-4ce6-87c4-ce5a95350120 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:41.561224 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a2db9-77c2-4ce6-87c4-ce5a95350120 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:41.561224 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a2db9-77c2-4ce6-87c4-ce5a95350120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:41.592660 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-c56a2db9-77c2-4ce6-87c4-ce5a95350120 None None] MFA Rules not processed for user `963f8c6c9f6d4b1aa076802c21021ae8`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:46:41.593899 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-c56a2db9-77c2-4ce6-87c4-ce5a95350120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:41.687754 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1135/2266] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:46:41 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:46:41.701939 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d9162-5753-48af-ba30-13e7d0090f91 service neutron] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:41.704145 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-504d9162-5753-48af-ba30-13e7d0090f91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:41.839835 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d9162-5753-48af-ba30-13e7d0090f91 service neutron] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:41.847490 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d667782-240d-4996-b547-658d53a81994 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:41.847490 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d667782-240d-4996-b547-658d53a81994 service nova] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:41.847490 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d667782-240d-4996-b547-658d53a81994 service nova] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:41.847869 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d667782-240d-4996-b547-658d53a81994 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:41.849780 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-3d667782-240d-4996-b547-658d53a81994 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:41.930068 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d667782-240d-4996-b547-658d53a81994 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:41.935650 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d667782-240d-4996-b547-658d53a81994 service nova] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:41.948690 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1132/2267] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:46:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:46:43.474677 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff31686-3c81-42b5-9abf-cb7a61b039bd None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:43.475141 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff31686-3c81-42b5-9abf-cb7a61b039bd None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:43.475222 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff31686-3c81-42b5-9abf-cb7a61b039bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:43.501978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-2ff31686-3c81-42b5-9abf-cb7a61b039bd None None] MFA Rules not processed for user `ce1b5129d46345ef99c69e95f9824c45`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:46:43.503485 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-2ff31686-3c81-42b5-9abf-cb7a61b039bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:43.513029 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1136/2268] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:46:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:46:45.523401 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f9a1a-d894-4850-aeec-1b0c85db1a77 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:45.523903 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f9a1a-d894-4850-aeec-1b0c85db1a77 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:45.524156 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f9a1a-d894-4850-aeec-1b0c85db1a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:45.558247 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-3e9f9a1a-d894-4850-aeec-1b0c85db1a77 None None] MFA Rules not processed for user `006f31b802f94549a517df90180dca36`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:46:45.559908 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-3e9f9a1a-d894-4850-aeec-1b0c85db1a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:45.601273 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:46:45.638237 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1133/2269] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:46:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:46:45.642668 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff31686-3c81-42b5-9abf-cb7a61b039bd None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:45.653553 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff31686-3c81-42b5-9abf-cb7a61b039bd None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:45.658023 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c71054d-4848-43c6-b053-a2799e9c43e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:45.658626 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71054d-4848-43c6-b053-a2799e9c43e5 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:45.658905 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71054d-4848-43c6-b053-a2799e9c43e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:45.659173 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71054d-4848-43c6-b053-a2799e9c43e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:45.659805 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c71054d-4848-43c6-b053-a2799e9c43e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:45.662193 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c71054d-4848-43c6-b053-a2799e9c43e5 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:45.695402 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1137/2270] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:46:45 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:46:45.698908 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f9a1a-d894-4850-aeec-1b0c85db1a77 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:45.713555 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f9a1a-d894-4850-aeec-1b0c85db1a77 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:45.776408 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0566abb-ab4a-46d5-adaa-2fcc1dcbde8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:45.777032 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0566abb-ab4a-46d5-adaa-2fcc1dcbde8d admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:45.777315 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0566abb-ab4a-46d5-adaa-2fcc1dcbde8d admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:45.777617 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0566abb-ab4a-46d5-adaa-2fcc1dcbde8d admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:45.778318 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0566abb-ab4a-46d5-adaa-2fcc1dcbde8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:45.780223 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0566abb-ab4a-46d5-adaa-2fcc1dcbde8d admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:45.805925 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1134/2271] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:46:45 2022] POST /identity/v3/users => generated 299 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:46:45.810931 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c71054d-4848-43c6-b053-a2799e9c43e5 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:45.823668 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c71054d-4848-43c6-b053-a2799e9c43e5 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:45.828990 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532ba79-8991-46ef-83e6-7cc17cc0e281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:45.829599 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532ba79-8991-46ef-83e6-7cc17cc0e281 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:45.829875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532ba79-8991-46ef-83e6-7cc17cc0e281 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:45.830150 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532ba79-8991-46ef-83e6-7cc17cc0e281 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:45.830821 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8532ba79-8991-46ef-83e6-7cc17cc0e281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:45.832470 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8532ba79-8991-46ef-83e6-7cc17cc0e281 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:45.862962 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1138/2272] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:46:45 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:46:45.867244 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0566abb-ab4a-46d5-adaa-2fcc1dcbde8d admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:45.879355 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0566abb-ab4a-46d5-adaa-2fcc1dcbde8d admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:45.884394 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07131dec-1dbc-428b-ac42-853b090d09fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:45.885169 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07131dec-1dbc-428b-ac42-853b090d09fc admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:45.885524 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07131dec-1dbc-428b-ac42-853b090d09fc admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:45.886282 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07131dec-1dbc-428b-ac42-853b090d09fc admin admin] PATH_INFO: `/v3/projects/3c5b125928c1443f86967c0f7c56fcec/users/41b0eeaee3724e7f8d3c4b63c298d5e6/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:45.905441 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07131dec-1dbc-428b-ac42-853b090d09fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c5b125928c1443f86967c0f7c56fcec, user_id=41b0eeaee3724e7f8d3c4b63c298d5e6, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:45.908282 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07131dec-1dbc-428b-ac42-853b090d09fc admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:45.924338 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1135/2273] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:46:45 2022] PUT /identity/v3/projects/3c5b125928c1443f86967c0f7c56fcec/users/41b0eeaee3724e7f8d3c4b63c298d5e6/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:46:45.928612 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532ba79-8991-46ef-83e6-7cc17cc0e281 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:45.940563 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532ba79-8991-46ef-83e6-7cc17cc0e281 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:46.007792 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:46:46.014873 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ecf5f9-ba10-4faf-8c06-819e4b34d6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:46.015716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ecf5f9-ba10-4faf-8c06-819e4b34d6b1 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:46.016224 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ecf5f9-ba10-4faf-8c06-819e4b34d6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:46.016592 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ecf5f9-ba10-4faf-8c06-819e4b34d6b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:46.017473 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ecf5f9-ba10-4faf-8c06-819e4b34d6b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:46.019315 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ecf5f9-ba10-4faf-8c06-819e4b34d6b1 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:46.034758 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1139/2274] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:46:45 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 23:46:46.036343 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07131dec-1dbc-428b-ac42-853b090d09fc admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:46.048045 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07131dec-1dbc-428b-ac42-853b090d09fc admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:46.053408 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d5251-77cc-410b-9ba1-20886aa58817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:46.055661 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d5251-77cc-410b-9ba1-20886aa58817 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:46.055661 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d5251-77cc-410b-9ba1-20886aa58817 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:46.055661 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d5251-77cc-410b-9ba1-20886aa58817 admin admin] PATH_INFO: `/v3/projects/3c5b125928c1443f86967c0f7c56fcec/users/41b0eeaee3724e7f8d3c4b63c298d5e6/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:46.057575 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9d5251-77cc-410b-9ba1-20886aa58817 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c5b125928c1443f86967c0f7c56fcec, user_id=41b0eeaee3724e7f8d3c4b63c298d5e6, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:46.060415 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9d5251-77cc-410b-9ba1-20886aa58817 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:46.075952 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1136/2275] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:46:46 2022] PUT /identity/v3/projects/3c5b125928c1443f86967c0f7c56fcec/users/41b0eeaee3724e7f8d3c4b63c298d5e6/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:46:51.310649 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2a051-a6dd-4561-88ae-05c0541f58f5 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:51.311127 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2a051-a6dd-4561-88ae-05c0541f58f5 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:51.311241 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2a051-a6dd-4561-88ae-05c0541f58f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:51.341408 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-f4e2a051-a6dd-4561-88ae-05c0541f58f5 None None] MFA Rules not processed for user `41b0eeaee3724e7f8d3c4b63c298d5e6`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:46:51.342784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-f4e2a051-a6dd-4561-88ae-05c0541f58f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:51.418394 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1140/2276] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:46:51 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:46:51.431094 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d5251-77cc-410b-9ba1-20886aa58817 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:51.433442 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-de9d5251-77cc-410b-9ba1-20886aa58817 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:51.495292 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d5251-77cc-410b-9ba1-20886aa58817 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:51.501010 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a4cd26-b0b3-49ce-9129-203e86d42f0c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c08c497a14c426f9cfe0dd8c8100a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:51.501912 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4cd26-b0b3-49ce-9129-203e86d42f0c service glance] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:51.502366 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4cd26-b0b3-49ce-9129-203e86d42f0c service glance] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:51.502909 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4cd26-b0b3-49ce-9129-203e86d42f0c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:51.548222 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:46:51.569172 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a4cd26-b0b3-49ce-9129-203e86d42f0c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:51.570897 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a4cd26-b0b3-49ce-9129-203e86d42f0c service glance] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:51.589258 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1137/2277] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:46:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 23:46:51.711080 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71077d17-3181-4ccd-8d67-b72cfd382afc None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:51.711439 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71077d17-3181-4ccd-8d67-b72cfd382afc None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:51.711635 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71077d17-3181-4ccd-8d67-b72cfd382afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:51.752993 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-71077d17-3181-4ccd-8d67-b72cfd382afc None None] MFA Rules not processed for user `d0a13dbffb4a49508b7fb661cf84e803`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:46:51.757320 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-71077d17-3181-4ccd-8d67-b72cfd382afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:51.846147 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1141/2278] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Aug 17 23:46:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:46:51.858497 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a4cd26-b0b3-49ce-9129-203e86d42f0c service glance] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:51.859915 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-e9a4cd26-b0b3-49ce-9129-203e86d42f0c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:51.939034 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a4cd26-b0b3-49ce-9129-203e86d42f0c service glance] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:51.944151 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183d4ec-bc15-48f5-bf6a-458aa40a77d6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee422ea90557444fabdc9c292865b9b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:51.944780 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183d4ec-bc15-48f5-bf6a-458aa40a77d6 service swift] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:51.945057 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183d4ec-bc15-48f5-bf6a-458aa40a77d6 service swift] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:51.945323 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183d4ec-bc15-48f5-bf6a-458aa40a77d6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:51.954329 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e183d4ec-bc15-48f5-bf6a-458aa40a77d6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:51.955903 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e183d4ec-bc15-48f5-bf6a-458aa40a77d6 service swift] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:51.964804 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1138/2279] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Aug 17 23:46:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 23:46:54.053399 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae151a-3791-4c82-9782-009217bcba79 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:54.054036 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae151a-3791-4c82-9782-009217bcba79 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:54.056095 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae151a-3791-4c82-9782-009217bcba79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:54.077384 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-87ae151a-3791-4c82-9782-009217bcba79 None None] MFA Rules not processed for user `cf6e346e96144628ac25c428be38870a`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:46:54.078665 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-87ae151a-3791-4c82-9782-009217bcba79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:54.168633 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1142/2280] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:46:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:46:56.619361 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183d4ec-bc15-48f5-bf6a-458aa40a77d6 service swift] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:46:56.620832 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-e183d4ec-bc15-48f5-bf6a-458aa40a77d6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:56.726208 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183d4ec-bc15-48f5-bf6a-458aa40a77d6 service swift] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:46:56.734785 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd390f1c-d9de-4e84-a70d-fa8a85d112f9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc57d911a56423e96c736bc96270ba8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:46:56.735457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd390f1c-d9de-4e84-a70d-fa8a85d112f9 service placement] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:46:56.735868 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd390f1c-d9de-4e84-a70d-fa8a85d112f9 service placement] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:46:56.736241 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd390f1c-d9de-4e84-a70d-fa8a85d112f9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:46:56.739312 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-fd390f1c-d9de-4e84-a70d-fa8a85d112f9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:46:56.752345 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd390f1c-d9de-4e84-a70d-fa8a85d112f9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:46:56.753999 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd390f1c-d9de-4e84-a70d-fa8a85d112f9 service placement] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:46:56.765345 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1139/2281] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:46:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:47:00.002493 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16068f23-e701-4c65-aeea-0209138e7397 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:47:00.003015 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16068f23-e701-4c65-aeea-0209138e7397 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:47:00.003116 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16068f23-e701-4c65-aeea-0209138e7397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:47:00.050669 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-16068f23-e701-4c65-aeea-0209138e7397 None None] MFA Rules not processed for user `21e06d9bb5a240459ef3298a990a42f2`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:47:00.052122 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-16068f23-e701-4c65-aeea-0209138e7397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:47:00.106009 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:47:00.183624 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1143/2282] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:46:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:47:03.251744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d20ad-6e07-4db3-b7a1-8fbd043b54c0 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:47:03.252340 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d20ad-6e07-4db3-b7a1-8fbd043b54c0 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:47:03.252605 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d20ad-6e07-4db3-b7a1-8fbd043b54c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:47:03.272850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-3c5d20ad-6e07-4db3-b7a1-8fbd043b54c0 None None] MFA Rules not processed for user `006f31b802f94549a517df90180dca36`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:47:03.274037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-3c5d20ad-6e07-4db3-b7a1-8fbd043b54c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:47:03.282481 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1140/2283] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:47:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:47:05.538827 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82e643-21e1-4aac-8f99-72d8be00ec87 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:47:05.539355 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82e643-21e1-4aac-8f99-72d8be00ec87 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:47:05.539355 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82e643-21e1-4aac-8f99-72d8be00ec87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:47:05.560244 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-2d82e643-21e1-4aac-8f99-72d8be00ec87 None None] MFA Rules not processed for user `d0a13dbffb4a49508b7fb661cf84e803`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:47:05.561448 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-2d82e643-21e1-4aac-8f99-72d8be00ec87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:47:05.568649 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1144/2284] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Aug 17 23:47:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:47:05.581955 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d20ad-6e07-4db3-b7a1-8fbd043b54c0 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:47:05.594942 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d20ad-6e07-4db3-b7a1-8fbd043b54c0 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:47:05.600304 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4fbbb8-8e28-42cc-9b55-d2d048d6dcf7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee422ea90557444fabdc9c292865b9b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:47:05.600927 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4fbbb8-8e28-42cc-9b55-d2d048d6dcf7 service swift] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:47:05.601204 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4fbbb8-8e28-42cc-9b55-d2d048d6dcf7 service swift] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:47:05.601478 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4fbbb8-8e28-42cc-9b55-d2d048d6dcf7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:47:05.610741 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4fbbb8-8e28-42cc-9b55-d2d048d6dcf7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:47:05.612468 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4fbbb8-8e28-42cc-9b55-d2d048d6dcf7 service swift] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:47:05.624417 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1141/2285] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Aug 17 23:47:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 23:47:15.555385 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=15 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:47:15.619264 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=83 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:47:34.477736 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d82e643-21e1-4aac-8f99-72d8be00ec87 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:47:34.542733 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d82e643-21e1-4aac-8f99-72d8be00ec87 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:47:34.547481 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec1c74-8288-4d27-a143-978b3a69f40b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:47:34.548235 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec1c74-8288-4d27-a143-978b3a69f40b service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:47:34.548626 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec1c74-8288-4d27-a143-978b3a69f40b service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:47:34.548966 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec1c74-8288-4d27-a143-978b3a69f40b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:47:34.550788 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-92ec1c74-8288-4d27-a143-978b3a69f40b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:47:34.655286 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ec1c74-8288-4d27-a143-978b3a69f40b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:47:34.656989 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ec1c74-8288-4d27-a143-978b3a69f40b service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:47:34.676115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1145/2286] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:47:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:47:44.664874 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=50 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:47:50.209671 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7719eb-467e-42cd-9129-5617a9515591 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:47:50.210225 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7719eb-467e-42cd-9129-5617a9515591 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:47:50.210403 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7719eb-467e-42cd-9129-5617a9515591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:47:50.230804 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-da7719eb-467e-42cd-9129-5617a9515591 None None] MFA Rules not processed for user `d0a13dbffb4a49508b7fb661cf84e803`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:47:50.232348 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-da7719eb-467e-42cd-9129-5617a9515591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:47:50.239494 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1142/2287] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Aug 17 23:47:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:47:50.261223 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec1c74-8288-4d27-a143-978b3a69f40b service neutron] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:47:50.272546 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec1c74-8288-4d27-a143-978b3a69f40b service neutron] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:47:50.278234 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cded3d4-990c-4a48-a32f-290ab6917626 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee422ea90557444fabdc9c292865b9b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:47:50.278919 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cded3d4-990c-4a48-a32f-290ab6917626 service swift] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:47:50.279263 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cded3d4-990c-4a48-a32f-290ab6917626 service swift] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:47:50.279597 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cded3d4-990c-4a48-a32f-290ab6917626 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:47:50.288917 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cded3d4-990c-4a48-a32f-290ab6917626 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:47:50.290633 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cded3d4-990c-4a48-a32f-290ab6917626 service swift] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:47:50.299787 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1146/2288] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Aug 17 23:47:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 23:47:54.454765 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7719eb-467e-42cd-9129-5617a9515591 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:47:54.456422 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-da7719eb-467e-42cd-9129-5617a9515591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:47:54.474721 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7719eb-467e-42cd-9129-5617a9515591 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:47:54.492341 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a40baa-3d56-4d1d-ab0c-9ff78f9dd713 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:47:54.492981 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a40baa-3d56-4d1d-ab0c-9ff78f9dd713 service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:47:54.493264 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a40baa-3d56-4d1d-ab0c-9ff78f9dd713 service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:47:54.493537 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a40baa-3d56-4d1d-ab0c-9ff78f9dd713 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:47:54.634394 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a40baa-3d56-4d1d-ab0c-9ff78f9dd713 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:47:54.637297 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a40baa-3d56-4d1d-ab0c-9ff78f9dd713 service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:47:54.678867 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1143/2289] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:47:54 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:47:54.915612 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cded3d4-990c-4a48-a32f-290ab6917626 service swift] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:47:54.925239 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cded3d4-990c-4a48-a32f-290ab6917626 service swift] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:47:54.928967 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920dc56a-e7c6-48fa-b56f-0dc7c03c8c21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:47:54.929727 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920dc56a-e7c6-48fa-b56f-0dc7c03c8c21 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:47:54.930006 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920dc56a-e7c6-48fa-b56f-0dc7c03c8c21 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:47:54.930276 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920dc56a-e7c6-48fa-b56f-0dc7c03c8c21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:47:54.940434 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920dc56a-e7c6-48fa-b56f-0dc7c03c8c21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:47:54.941874 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920dc56a-e7c6-48fa-b56f-0dc7c03c8c21 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:47:54.952431 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1147/2290] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:47:54 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:48:04.658392 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=33 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:48:04.948246 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=12 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:48:08.745222 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f6b31-109c-49b8-bde9-bc77499a2fae None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:48:08.745768 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f6b31-109c-49b8-bde9-bc77499a2fae None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:48:08.745768 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f6b31-109c-49b8-bde9-bc77499a2fae None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:48:08.747046 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1144/2291] 10.222.0.26 () {54 vars in 921 bytes} [Wed Aug 17 23:48:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:48:08.752826 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34965c-ed81-4fde-a258-20c15f29c70a None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:48:08.753297 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34965c-ed81-4fde-a258-20c15f29c70a None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:48:08.753463 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34965c-ed81-4fde-a258-20c15f29c70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:48:08.772935 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-ff34965c-ed81-4fde-a258-20c15f29c70a None None] MFA Rules not processed for user `ec1dcbe43b0749c3bcc4af9100d771d1`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:48:08.775115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-ff34965c-ed81-4fde-a258-20c15f29c70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:48:08.864783 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1148/2292] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Aug 17 23:48:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:48:08.873754 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f6b31-109c-49b8-bde9-bc77499a2fae None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:48:08.885321 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f6b31-109c-49b8-bde9-bc77499a2fae None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:48:08.889027 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f797786-fc55-4d36-aae4-1a11792b7cb6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c08c497a14c426f9cfe0dd8c8100a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:48:08.889634 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f797786-fc55-4d36-aae4-1a11792b7cb6 service glance] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:48:08.889995 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f797786-fc55-4d36-aae4-1a11792b7cb6 service glance] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:48:08.890256 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f797786-fc55-4d36-aae4-1a11792b7cb6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:48:08.906437 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f797786-fc55-4d36-aae4-1a11792b7cb6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:48:08.907867 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f797786-fc55-4d36-aae4-1a11792b7cb6 service glance] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:48:08.920304 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1145/2293] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:48:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:48:08.992321 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7666b4-1af7-4c26-9fb1-c958bed8591b None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:48:08.992606 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7666b4-1af7-4c26-9fb1-c958bed8591b None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:48:08.992877 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7666b4-1af7-4c26-9fb1-c958bed8591b None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:48:08.994222 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1149/2294] 10.222.0.26 () {54 vars in 921 bytes} [Wed Aug 17 23:48:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:48:08.998899 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f4b49-809f-448c-9382-7f6b4f150641 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:48:08.999128 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f4b49-809f-448c-9382-7f6b4f150641 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:48:08.999402 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f4b49-809f-448c-9382-7f6b4f150641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:48:09.019116 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-489f4b49-809f-448c-9382-7f6b4f150641 None None] MFA Rules not processed for user `16e2758f1cd144a98b98d9be0fa46ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:48:09.020352 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-489f4b49-809f-448c-9382-7f6b4f150641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:48:09.027420 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1146/2295] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Aug 17 23:48:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:48:09.039509 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7666b4-1af7-4c26-9fb1-c958bed8591b None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:48:09.050728 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7666b4-1af7-4c26-9fb1-c958bed8591b None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:48:09.054324 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f6bf7-f5da-4a8d-893e-e22cd84fdef5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:48:09.054906 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f6bf7-f5da-4a8d-893e-e22cd84fdef5 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:48:09.055200 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f6bf7-f5da-4a8d-893e-e22cd84fdef5 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:48:09.055466 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f6bf7-f5da-4a8d-893e-e22cd84fdef5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:48:09.062494 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f6bf7-f5da-4a8d-893e-e22cd84fdef5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:48:09.063923 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f6bf7-f5da-4a8d-893e-e22cd84fdef5 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:48:09.072078 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1150/2296] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:48:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:48:09.154211 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a03d3ce-1262-4e31-a4dc-4c968e77bce9 None None] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:48:09.154496 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a03d3ce-1262-4e31-a4dc-4c968e77bce9 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:48:09.154771 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a03d3ce-1262-4e31-a4dc-4c968e77bce9 None None] PATH_INFO: `/` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:48:09.156209 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1147/2297] 10.222.0.26 () {54 vars in 921 bytes} [Wed Aug 17 23:48:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:48:09.161750 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ac0c1-af1b-4014-b012-1e8ed1c1dc3f None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:48:09.162029 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ac0c1-af1b-4014-b012-1e8ed1c1dc3f None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:48:09.162294 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ac0c1-af1b-4014-b012-1e8ed1c1dc3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:48:09.182399 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-270ac0c1-af1b-4014-b012-1e8ed1c1dc3f None None] MFA Rules not processed for user `ec1dcbe43b0749c3bcc4af9100d771d1`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:48:09.183625 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-270ac0c1-af1b-4014-b012-1e8ed1c1dc3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:48:09.192829 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1151/2298] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Aug 17 23:48:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:48:09.203563 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a03d3ce-1262-4e31-a4dc-4c968e77bce9 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:48:09.215041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a03d3ce-1262-4e31-a4dc-4c968e77bce9 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:48:09.218611 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95e7da8-1fad-4a08-a31d-ca83a4ac85c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:48:09.219215 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e7da8-1fad-4a08-a31d-ca83a4ac85c7 service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:48:09.219494 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e7da8-1fad-4a08-a31d-ca83a4ac85c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:48:09.219765 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e7da8-1fad-4a08-a31d-ca83a4ac85c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:48:09.230558 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95e7da8-1fad-4a08-a31d-ca83a4ac85c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:48:09.232098 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95e7da8-1fad-4a08-a31d-ca83a4ac85c7 service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:48:09.242718 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1148/2299] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:48:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:48:12.311813 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ac0c1-af1b-4014-b012-1e8ed1c1dc3f None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:48:12.321720 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ac0c1-af1b-4014-b012-1e8ed1c1dc3f None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:48:12.325303 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840f509-6a11-49b6-8b63-cea731720fad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:48:12.325886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840f509-6a11-49b6-8b63-cea731720fad service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:48:12.326186 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840f509-6a11-49b6-8b63-cea731720fad service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:48:12.326452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840f509-6a11-49b6-8b63-cea731720fad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:48:12.328186 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-8840f509-6a11-49b6-8b63-cea731720fad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:48:12.341581 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8840f509-6a11-49b6-8b63-cea731720fad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:48:12.342998 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8840f509-6a11-49b6-8b63-cea731720fad service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:48:12.353715 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1152/2300] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:48:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:48:13.539410 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95e7da8-1fad-4a08-a31d-ca83a4ac85c7 service neutron] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:48:13.547872 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95e7da8-1fad-4a08-a31d-ca83a4ac85c7 service neutron] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:48:13.551434 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b2f8c-9f96-4cec-8bc3-a1e6468cd8cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:48:13.552047 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b2f8c-9f96-4cec-8bc3-a1e6468cd8cd service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:48:13.552327 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b2f8c-9f96-4cec-8bc3-a1e6468cd8cd service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:48:13.552597 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b2f8c-9f96-4cec-8bc3-a1e6468cd8cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:48:13.554928 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-0c4b2f8c-9f96-4cec-8bc3-a1e6468cd8cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:48:13.573586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4b2f8c-9f96-4cec-8bc3-a1e6468cd8cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:48:13.575004 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4b2f8c-9f96-4cec-8bc3-a1e6468cd8cd service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:48:13.586521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1149/2301] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:48:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:48:18.435237 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf1b580-55b3-48cd-bc3f-2e10de29c833 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:48:18.435689 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf1b580-55b3-48cd-bc3f-2e10de29c833 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:48:18.435798 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf1b580-55b3-48cd-bc3f-2e10de29c833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:48:18.455571 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-3bf1b580-55b3-48cd-bc3f-2e10de29c833 None None] MFA Rules not processed for user `d0a13dbffb4a49508b7fb661cf84e803`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:48:18.456833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-3bf1b580-55b3-48cd-bc3f-2e10de29c833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:48:18.463910 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1153/2302] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Aug 17 23:48:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:48:18.475714 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b2f8c-9f96-4cec-8bc3-a1e6468cd8cd service neutron] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:48:18.486642 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b2f8c-9f96-4cec-8bc3-a1e6468cd8cd service neutron] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:48:18.491507 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed40d3ff-c985-4c4b-9b03-f8bab0a2b47a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee422ea90557444fabdc9c292865b9b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:48:18.492673 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed40d3ff-c985-4c4b-9b03-f8bab0a2b47a service swift] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:48:18.492949 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed40d3ff-c985-4c4b-9b03-f8bab0a2b47a service swift] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:48:18.493220 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed40d3ff-c985-4c4b-9b03-f8bab0a2b47a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:48:18.501123 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed40d3ff-c985-4c4b-9b03-f8bab0a2b47a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:48:18.502732 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed40d3ff-c985-4c4b-9b03-f8bab0a2b47a service swift] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:48:18.511466 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1150/2303] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Aug 17 23:48:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 17 23:48:26.086134 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf1b580-55b3-48cd-bc3f-2e10de29c833 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:48:26.098828 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf1b580-55b3-48cd-bc3f-2e10de29c833 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:48:26.104962 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb21e6-0d88-46f7-b20d-8782cac640a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:48:26.105634 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb21e6-0d88-46f7-b20d-8782cac640a9 service nova] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:48:26.105916 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb21e6-0d88-46f7-b20d-8782cac640a9 service nova] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:48:26.106182 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb21e6-0d88-46f7-b20d-8782cac640a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:48:26.107889 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-66cb21e6-0d88-46f7-b20d-8782cac640a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:48:26.165750 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cb21e6-0d88-46f7-b20d-8782cac640a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:48:26.167284 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cb21e6-0d88-46f7-b20d-8782cac640a9 service nova] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:48:26.176547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1154/2304] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:48:26 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:48:28.507635 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=27 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:48:28.945815 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed40d3ff-c985-4c4b-9b03-f8bab0a2b47a service swift] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:48:28.947120 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-ed40d3ff-c985-4c4b-9b03-f8bab0a2b47a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:48:28.962575 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed40d3ff-c985-4c4b-9b03-f8bab0a2b47a service swift] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:48:28.969647 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba919518-0770-42bd-ba51-7e04c6d1130f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:48:28.970289 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba919518-0770-42bd-ba51-7e04c6d1130f service nova] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:48:28.970579 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba919518-0770-42bd-ba51-7e04c6d1130f service nova] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:48:28.970887 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba919518-0770-42bd-ba51-7e04c6d1130f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:48:28.972714 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-ba919518-0770-42bd-ba51-7e04c6d1130f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:48:29.051087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba919518-0770-42bd-ba51-7e04c6d1130f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:48:29.052631 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba919518-0770-42bd-ba51-7e04c6d1130f service nova] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:48:29.061707 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1151/2305] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:48:28 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:48:36.172256 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=77 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:48:39.057624 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=28 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:48:54.132742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cac43e-6107-46c9-b3e3-169e0cbea0b9 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:48:54.133210 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cac43e-6107-46c9-b3e3-169e0cbea0b9 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:48:54.133303 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cac43e-6107-46c9-b3e3-169e0cbea0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:48:54.151996 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-c3cac43e-6107-46c9-b3e3-169e0cbea0b9 None None] MFA Rules not processed for user `d0a13dbffb4a49508b7fb661cf84e803`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:48:54.153203 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-c3cac43e-6107-46c9-b3e3-169e0cbea0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:48:54.160318 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1155/2306] 10.222.0.26 () {58 vars in 967 bytes} [Wed Aug 17 23:48:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:48:54.182877 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba919518-0770-42bd-ba51-7e04c6d1130f service nova] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:48:54.193949 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba919518-0770-42bd-ba51-7e04c6d1130f service nova] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:48:54.199089 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ad9e81-7e66-4007-af4f-b644bfd82217 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee422ea90557444fabdc9c292865b9b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:48:54.199626 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad9e81-7e66-4007-af4f-b644bfd82217 service swift] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:48:54.199903 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad9e81-7e66-4007-af4f-b644bfd82217 service swift] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:48:54.200203 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad9e81-7e66-4007-af4f-b644bfd82217 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:48:54.207986 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ad9e81-7e66-4007-af4f-b644bfd82217 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:48:54.209570 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ad9e81-7e66-4007-af4f-b644bfd82217 service swift] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:48:54.217539 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1152/2307] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Aug 17 23:48:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 23:49:04.146890 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=3 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:49:04.214273 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=6 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:49:04.959423 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cac43e-6107-46c9-b3e3-169e0cbea0b9 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:49:04.968836 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cac43e-6107-46c9-b3e3-169e0cbea0b9 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:49:04.972442 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346a2dfe-98d9-4c72-81e8-71f981819012 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:49:04.973034 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a2dfe-98d9-4c72-81e8-71f981819012 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:49:04.973310 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a2dfe-98d9-4c72-81e8-71f981819012 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:49:04.973576 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a2dfe-98d9-4c72-81e8-71f981819012 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:49:04.975311 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-346a2dfe-98d9-4c72-81e8-71f981819012 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:49:04.984918 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346a2dfe-98d9-4c72-81e8-71f981819012 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:49:04.986450 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346a2dfe-98d9-4c72-81e8-71f981819012 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:49:04.995226 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1156/2308] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:49:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:49:07.551497 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b4098-63ad-4974-8bd1-f5705db05028 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:49:07.552046 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b4098-63ad-4974-8bd1-f5705db05028 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:49:07.552212 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b4098-63ad-4974-8bd1-f5705db05028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:49:07.571040 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-7e7b4098-63ad-4974-8bd1-f5705db05028 None None] MFA Rules not processed for user `c67dec924b7e4f0aa0874e6e21879889`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:49:07.572499 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-7e7b4098-63ad-4974-8bd1-f5705db05028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:49:07.579237 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1153/2309] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:49:07 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:49:13.667814 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346a2dfe-98d9-4c72-81e8-71f981819012 service neutron] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:49:13.684493 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346a2dfe-98d9-4c72-81e8-71f981819012 service neutron] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:49:13.689960 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f6ece-0d43-4272-a683-1a95f0e326ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:49:13.690559 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f6ece-0d43-4272-a683-1a95f0e326ab service nova] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:49:13.690838 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f6ece-0d43-4272-a683-1a95f0e326ab service nova] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:49:13.691110 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f6ece-0d43-4272-a683-1a95f0e326ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:49:13.692836 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-9b1f6ece-0d43-4272-a683-1a95f0e326ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:49:13.705015 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1f6ece-0d43-4272-a683-1a95f0e326ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:49:13.706660 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1f6ece-0d43-4272-a683-1a95f0e326ab service nova] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:49:13.717240 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1157/2310] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:49:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:49:17.565798 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=3 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:49:23.712754 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=12 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:49:37.734618 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b4098-63ad-4974-8bd1-f5705db05028 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:49:37.747726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b4098-63ad-4974-8bd1-f5705db05028 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:49:37.753003 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a70b565-1713-4f61-a30c-8c998ab7e4e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:49:37.753587 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70b565-1713-4f61-a30c-8c998ab7e4e4 service nova] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:49:37.753861 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70b565-1713-4f61-a30c-8c998ab7e4e4 service nova] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:49:37.754129 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70b565-1713-4f61-a30c-8c998ab7e4e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:49:37.755817 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-6a70b565-1713-4f61-a30c-8c998ab7e4e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:49:37.765745 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a70b565-1713-4f61-a30c-8c998ab7e4e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:49:37.767308 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a70b565-1713-4f61-a30c-8c998ab7e4e4 service nova] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:49:37.776460 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1154/2311] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:49:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:49:41.207040 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f6ece-0d43-4272-a683-1a95f0e326ab service nova] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:49:41.220336 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f6ece-0d43-4272-a683-1a95f0e326ab service nova] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:49:41.226521 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa725ad-5a67-4047-8d95-f95126d330b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:49:41.227473 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa725ad-5a67-4047-8d95-f95126d330b1 service nova] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:49:41.227894 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa725ad-5a67-4047-8d95-f95126d330b1 service nova] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:49:41.228256 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa725ad-5a67-4047-8d95-f95126d330b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:49:41.230117 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-4aa725ad-5a67-4047-8d95-f95126d330b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:49:41.247072 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa725ad-5a67-4047-8d95-f95126d330b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:49:41.248817 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa725ad-5a67-4047-8d95-f95126d330b1 service nova] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:49:41.261315 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1158/2312] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:49:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:49:47.771979 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=6 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:49:51.255445 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=6 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:49:52.658647 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1ce3e-8c68-402a-b925-db1cc5d3c6ff None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:49:52.659466 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1ce3e-8c68-402a-b925-db1cc5d3c6ff None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:49:52.659739 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1ce3e-8c68-402a-b925-db1cc5d3c6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:49:52.679021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-d8f1ce3e-8c68-402a-b925-db1cc5d3c6ff None None] MFA Rules not processed for user `21e06d9bb5a240459ef3298a990a42f2`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:49:52.680286 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-d8f1ce3e-8c68-402a-b925-db1cc5d3c6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:49:52.687457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1155/2313] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:49:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:49:52.994874 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa725ad-5a67-4047-8d95-f95126d330b1 service nova] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:49:53.004110 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa725ad-5a67-4047-8d95-f95126d330b1 service nova] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:49:53.007942 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25cb20-1998-499c-8797-771a0ad2a770 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:49:53.008631 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25cb20-1998-499c-8797-771a0ad2a770 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:49:53.008977 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25cb20-1998-499c-8797-771a0ad2a770 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:49:53.009821 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25cb20-1998-499c-8797-771a0ad2a770 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:49:53.011639 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-bb25cb20-1998-499c-8797-771a0ad2a770 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:49:53.021379 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25cb20-1998-499c-8797-771a0ad2a770 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:49:53.022796 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25cb20-1998-499c-8797-771a0ad2a770 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:49:53.031198 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1159/2314] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:49:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:49:54.348937 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1ce3e-8c68-402a-b925-db1cc5d3c6ff None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:49:54.350359 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-d8f1ce3e-8c68-402a-b925-db1cc5d3c6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:49:54.365198 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1ce3e-8c68-402a-b925-db1cc5d3c6ff None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:49:54.370741 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe3bd53-6363-4a39-8fd2-d8e32f3c4b9e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc57d911a56423e96c736bc96270ba8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:49:54.371328 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe3bd53-6363-4a39-8fd2-d8e32f3c4b9e service placement] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:49:54.371601 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe3bd53-6363-4a39-8fd2-d8e32f3c4b9e service placement] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:49:54.371867 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe3bd53-6363-4a39-8fd2-d8e32f3c4b9e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:49:54.373525 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-5fe3bd53-6363-4a39-8fd2-d8e32f3c4b9e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:49:54.384877 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe3bd53-6363-4a39-8fd2-d8e32f3c4b9e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:49:54.386454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe3bd53-6363-4a39-8fd2-d8e32f3c4b9e service placement] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:49:54.396404 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1156/2315] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:49:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:50:03.027825 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=6 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:50:04.392201 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=9 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:50:06.462874 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25cb20-1998-499c-8797-771a0ad2a770 service neutron] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:50:06.476880 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25cb20-1998-499c-8797-771a0ad2a770 service neutron] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:50:06.482869 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597b8a2-e474-4533-a17a-27864139a088 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:50:06.483461 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597b8a2-e474-4533-a17a-27864139a088 service nova] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:06.483734 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597b8a2-e474-4533-a17a-27864139a088 service nova] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:06.484021 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597b8a2-e474-4533-a17a-27864139a088 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:06.486142 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-c597b8a2-e474-4533-a17a-27864139a088 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:50:06.500011 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c597b8a2-e474-4533-a17a-27864139a088 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:50:06.501590 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c597b8a2-e474-4533-a17a-27864139a088 service nova] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:50:06.512387 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1160/2316] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:50:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:50:12.848211 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ddd849-81b9-4167-8ed0-c9ec5526ddc4 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:12.848873 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ddd849-81b9-4167-8ed0-c9ec5526ddc4 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:12.849171 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ddd849-81b9-4167-8ed0-c9ec5526ddc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:12.870023 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-27ddd849-81b9-4167-8ed0-c9ec5526ddc4 None None] MFA Rules not processed for user `2da686247e7f490ab7924d0456da60d5`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:50:12.871412 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-27ddd849-81b9-4167-8ed0-c9ec5526ddc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:50:12.878372 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1157/2317] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:50:12 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:50:16.507785 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=6 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:50:22.864873 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=3 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:50:34.655910 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db406ac0-fda3-45b4-9674-1b84757cfc77 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:34.656426 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db406ac0-fda3-45b4-9674-1b84757cfc77 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:34.656528 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db406ac0-fda3-45b4-9674-1b84757cfc77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:34.674838 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-db406ac0-fda3-45b4-9674-1b84757cfc77 None None] MFA Rules not processed for user `cf6e346e96144628ac25c428be38870a`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:50:34.676052 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-db406ac0-fda3-45b4-9674-1b84757cfc77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:50:34.682839 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1161/2318] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:50:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:50:35.011204 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ddd849-81b9-4167-8ed0-c9ec5526ddc4 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:50:35.021241 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ddd849-81b9-4167-8ed0-c9ec5526ddc4 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:50:35.024680 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5451e52-a8a0-4cef-834f-3612294b54cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:50:35.025296 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5451e52-a8a0-4cef-834f-3612294b54cf service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:35.025575 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5451e52-a8a0-4cef-834f-3612294b54cf service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:35.025876 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5451e52-a8a0-4cef-834f-3612294b54cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:35.027905 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-e5451e52-a8a0-4cef-834f-3612294b54cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:50:35.038403 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5451e52-a8a0-4cef-834f-3612294b54cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:50:35.039832 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5451e52-a8a0-4cef-834f-3612294b54cf service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:50:35.047653 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1158/2319] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:50:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:50:44.221845 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db406ac0-fda3-45b4-9674-1b84757cfc77 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:50:44.231206 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db406ac0-fda3-45b4-9674-1b84757cfc77 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:50:44.234680 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d8b418-4d34-4ae9-81e6-0fe015dd3e09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:50:44.235265 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d8b418-4d34-4ae9-81e6-0fe015dd3e09 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:44.235537 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d8b418-4d34-4ae9-81e6-0fe015dd3e09 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:44.235802 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d8b418-4d34-4ae9-81e6-0fe015dd3e09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:44.237579 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-04d8b418-4d34-4ae9-81e6-0fe015dd3e09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:50:44.298736 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d8b418-4d34-4ae9-81e6-0fe015dd3e09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:50:44.300135 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d8b418-4d34-4ae9-81e6-0fe015dd3e09 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:50:44.310161 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1162/2320] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:50:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:50:45.044192 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=6 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:50:47.577281 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5451e52-a8a0-4cef-834f-3612294b54cf service neutron] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:50:47.591941 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5451e52-a8a0-4cef-834f-3612294b54cf service neutron] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:50:47.597103 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b2a7cf-8015-4b51-b281-e1dad8461f05 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc57d911a56423e96c736bc96270ba8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:50:47.597702 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2a7cf-8015-4b51-b281-e1dad8461f05 service placement] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:47.597979 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2a7cf-8015-4b51-b281-e1dad8461f05 service placement] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:47.598799 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2a7cf-8015-4b51-b281-e1dad8461f05 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:47.600658 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-76b2a7cf-8015-4b51-b281-e1dad8461f05 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:50:47.612267 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b2a7cf-8015-4b51-b281-e1dad8461f05 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:50:47.613850 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b2a7cf-8015-4b51-b281-e1dad8461f05 service placement] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:50:47.626390 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1159/2321] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:50:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 23:50:47.635614 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d8b418-4d34-4ae9-81e6-0fe015dd3e09 service neutron] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:50:47.637164 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-04d8b418-4d34-4ae9-81e6-0fe015dd3e09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:50:47.650509 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d8b418-4d34-4ae9-81e6-0fe015dd3e09 service neutron] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:50:47.656940 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f0fcb-c221-46d7-8c37-6a21c381e015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:50:47.657568 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f0fcb-c221-46d7-8c37-6a21c381e015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:47.657848 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f0fcb-c221-46d7-8c37-6a21c381e015 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:47.658124 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f0fcb-c221-46d7-8c37-6a21c381e015 admin admin] PATH_INFO: `/v3/users/c67dec924b7e4f0aa0874e6e21879889` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:47.659377 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853f0fcb-c221-46d7-8c37-6a21c381e015 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c67dec924b7e4f0aa0874e6e21879889)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:50:47.661412 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853f0fcb-c221-46d7-8c37-6a21c381e015 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:50:47.707385 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-853f0fcb-c221-46d7-8c37-6a21c381e015 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c67dec924b7e4f0aa0874e6e21879889'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:50:47.707983 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-853f0fcb-c221-46d7-8c37-6a21c381e015 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c67dec924b7e4f0aa0874e6e21879889'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:50:47.720165 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-853f0fcb-c221-46d7-8c37-6a21c381e015 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c67dec924b7e4f0aa0874e6e21879889'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:50:47.726952 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-853f0fcb-c221-46d7-8c37-6a21c381e015 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c67dec924b7e4f0aa0874e6e21879889'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:50:47.727588 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-853f0fcb-c221-46d7-8c37-6a21c381e015 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c67dec924b7e4f0aa0874e6e21879889'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:50:47.739617 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1163/2322] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:50:47 2022] DELETE /identity/v3/users/c67dec924b7e4f0aa0874e6e21879889 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:50:47.911084 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b2a7cf-8015-4b51-b281-e1dad8461f05 service placement] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:50:47.923775 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b2a7cf-8015-4b51-b281-e1dad8461f05 service placement] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:50:47.980107 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd431a-5f36-4f30-b3a6-b275ccdfea4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:50:47.980728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd431a-5f36-4f30-b3a6-b275ccdfea4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:47.981004 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd431a-5f36-4f30-b3a6-b275ccdfea4f admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:47.981271 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd431a-5f36-4f30-b3a6-b275ccdfea4f admin admin] PATH_INFO: `/v3/projects/dbc0fe79b67f4446b1a4c36c068f9b00` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:47.982372 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd431a-5f36-4f30-b3a6-b275ccdfea4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbc0fe79b67f4446b1a4c36c068f9b00)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:50:47.984298 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd431a-5f36-4f30-b3a6-b275ccdfea4f admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:50:48.028429 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-25dd431a-5f36-4f30-b3a6-b275ccdfea4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbc0fe79b67f4446b1a4c36c068f9b00', 'request_id': 'req-25dd431a-5f36-4f30-b3a6-b275ccdfea4f'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:50:48.033763 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-25dd431a-5f36-4f30-b3a6-b275ccdfea4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbc0fe79b67f4446b1a4c36c068f9b00', 'request_id': 'req-25dd431a-5f36-4f30-b3a6-b275ccdfea4f'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:50:48.041722 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-25dd431a-5f36-4f30-b3a6-b275ccdfea4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbc0fe79b67f4446b1a4c36c068f9b00', 'request_id': 'req-25dd431a-5f36-4f30-b3a6-b275ccdfea4f'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:50:48.053083 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-25dd431a-5f36-4f30-b3a6-b275ccdfea4f admin admin] The token cache is being invalidate because project dbc0fe79b67f4446b1a4c36c068f9b00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:50:48.053377 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-25dd431a-5f36-4f30-b3a6-b275ccdfea4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:50:48.054070 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-25dd431a-5f36-4f30-b3a6-b275ccdfea4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:50:48.055667 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1160/2323] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:50:47 2022] DELETE /identity/v3/projects/dbc0fe79b67f4446b1a4c36c068f9b00 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:50:48.061229 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c4375-1962-4b44-bda7-2a3a1b3c36d9 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:48.061539 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c4375-1962-4b44-bda7-2a3a1b3c36d9 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:48.061814 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c4375-1962-4b44-bda7-2a3a1b3c36d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:48.081729 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-859c4375-1962-4b44-bda7-2a3a1b3c36d9 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:50:48.082932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-859c4375-1962-4b44-bda7-2a3a1b3c36d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:50:48.131764 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:50:48.141561 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1164/2324] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:50:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:50:48.148974 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547b19a-4afd-4c86-baa6-65400c913ea3 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:48.149262 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547b19a-4afd-4c86-baa6-65400c913ea3 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:48.149555 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547b19a-4afd-4c86-baa6-65400c913ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:48.171277 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-3547b19a-4afd-4c86-baa6-65400c913ea3 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:50:48.172433 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-3547b19a-4afd-4c86-baa6-65400c913ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:50:48.180223 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1161/2325] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:50:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:50:48.185056 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c4375-1962-4b44-bda7-2a3a1b3c36d9 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:50:48.194744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c4375-1962-4b44-bda7-2a3a1b3c36d9 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:50:48.199314 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eecc5b6-5997-45e0-a30d-a90b3180863c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:50:48.199934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecc5b6-5997-45e0-a30d-a90b3180863c admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:48.200246 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecc5b6-5997-45e0-a30d-a90b3180863c admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:48.200520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecc5b6-5997-45e0-a30d-a90b3180863c admin admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:48.201168 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eecc5b6-5997-45e0-a30d-a90b3180863c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:50:48.201472 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eecc5b6-5997-45e0-a30d-a90b3180863c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:50:48.203076 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eecc5b6-5997-45e0-a30d-a90b3180863c admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:50:48.214346 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1165/2326] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:50:48.218653 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3547b19a-4afd-4c86-baa6-65400c913ea3 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:50:48.227690 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3547b19a-4afd-4c86-baa6-65400c913ea3 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:50:48.232768 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3cc82b-ac0b-4ba9-a4c4-6bb004d70429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:50:48.233348 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cc82b-ac0b-4ba9-a4c4-6bb004d70429 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:48.233627 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cc82b-ac0b-4ba9-a4c4-6bb004d70429 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:48.233894 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cc82b-ac0b-4ba9-a4c4-6bb004d70429 admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:48.234510 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3cc82b-ac0b-4ba9-a4c4-6bb004d70429 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:50:48.236419 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3cc82b-ac0b-4ba9-a4c4-6bb004d70429 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:50:48.262797 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1162/2327] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:50:48 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:50:48.267101 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eecc5b6-5997-45e0-a30d-a90b3180863c admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:50:48.277030 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eecc5b6-5997-45e0-a30d-a90b3180863c admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:50:48.333351 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611299db-12d6-4226-bdf4-ad7fcf61da8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:50:48.333941 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611299db-12d6-4226-bdf4-ad7fcf61da8e admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:48.334217 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611299db-12d6-4226-bdf4-ad7fcf61da8e admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:48.334485 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611299db-12d6-4226-bdf4-ad7fcf61da8e admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:48.335088 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611299db-12d6-4226-bdf4-ad7fcf61da8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:50:48.337020 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611299db-12d6-4226-bdf4-ad7fcf61da8e admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:50:48.357484 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1166/2328] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:50:48 2022] POST /identity/v3/users => generated 283 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:50:48.361601 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3cc82b-ac0b-4ba9-a4c4-6bb004d70429 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:50:48.371988 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3cc82b-ac0b-4ba9-a4c4-6bb004d70429 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:50:48.376510 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79fb0f-e1e1-4278-b8ad-2d9bfce0d56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:50:48.377102 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79fb0f-e1e1-4278-b8ad-2d9bfce0d56b admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:48.377381 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79fb0f-e1e1-4278-b8ad-2d9bfce0d56b admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:48.377652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79fb0f-e1e1-4278-b8ad-2d9bfce0d56b admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:48.378292 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee79fb0f-e1e1-4278-b8ad-2d9bfce0d56b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:50:48.379885 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee79fb0f-e1e1-4278-b8ad-2d9bfce0d56b admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:50:48.390008 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1163/2329] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:50:48.393746 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611299db-12d6-4226-bdf4-ad7fcf61da8e admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:50:48.404132 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611299db-12d6-4226-bdf4-ad7fcf61da8e admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:50:48.408743 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357b9461-d3c8-4466-8060-ce0d18da57ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:50:48.409331 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b9461-d3c8-4466-8060-ce0d18da57ea admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:48.409607 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b9461-d3c8-4466-8060-ce0d18da57ea admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:48.409876 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b9461-d3c8-4466-8060-ce0d18da57ea admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:48.410658 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357b9461-d3c8-4466-8060-ce0d18da57ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:50:48.412530 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357b9461-d3c8-4466-8060-ce0d18da57ea admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:50:48.422603 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1167/2330] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 23:50:48.426656 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79fb0f-e1e1-4278-b8ad-2d9bfce0d56b admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:50:48.436910 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79fb0f-e1e1-4278-b8ad-2d9bfce0d56b admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:50:48.443652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d69dd-9e76-46b3-9f94-e9d1ff6de560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:50:48.443652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d69dd-9e76-46b3-9f94-e9d1ff6de560 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:48.443652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d69dd-9e76-46b3-9f94-e9d1ff6de560 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:48.443652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d69dd-9e76-46b3-9f94-e9d1ff6de560 admin admin] PATH_INFO: `/v3/projects/0ad5c7721e2b44b7bc33221137152d56/users/ce15e9a8d1e245aa94dbe479c76cc2d9/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:48.460124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d69dd-9e76-46b3-9f94-e9d1ff6de560 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ad5c7721e2b44b7bc33221137152d56, user_id=ce15e9a8d1e245aa94dbe479c76cc2d9, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:50:48.462797 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d69dd-9e76-46b3-9f94-e9d1ff6de560 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:50:48.476504 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1164/2331] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:50:48 2022] PUT /identity/v3/projects/0ad5c7721e2b44b7bc33221137152d56/users/ce15e9a8d1e245aa94dbe479c76cc2d9/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:50:48.487939 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb739c2b-bcc8-43d9-ae43-1c1cf50036ba None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:48.488347 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb739c2b-bcc8-43d9-ae43-1c1cf50036ba None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:48.488630 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb739c2b-bcc8-43d9-ae43-1c1cf50036ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:48.517519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-fb739c2b-bcc8-43d9-ae43-1c1cf50036ba None None] MFA Rules not processed for user `ce15e9a8d1e245aa94dbe479c76cc2d9`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:50:48.518736 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-fb739c2b-bcc8-43d9-ae43-1c1cf50036ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:50:48.587756 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1168/2332] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:50:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:50:48.598550 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d69dd-9e76-46b3-9f94-e9d1ff6de560 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:50:48.600318 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-eb5d69dd-9e76-46b3-9f94-e9d1ff6de560 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:50:48.614268 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:50:48.658121 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d69dd-9e76-46b3-9f94-e9d1ff6de560 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:50:48.661751 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a65b0ab-a721-4d21-b5d5-193da547610e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:50:48.662600 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65b0ab-a721-4d21-b5d5-193da547610e service cinder] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:48.662879 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65b0ab-a721-4d21-b5d5-193da547610e service cinder] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:48.663151 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65b0ab-a721-4d21-b5d5-193da547610e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:48.671153 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a65b0ab-a721-4d21-b5d5-193da547610e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:50:48.672656 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a65b0ab-a721-4d21-b5d5-193da547610e service cinder] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:50:48.681237 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1165/2333] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:50:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:50:57.033053 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb739c2b-bcc8-43d9-ae43-1c1cf50036ba None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:50:57.113844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:50:57.120201 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb739c2b-bcc8-43d9-ae43-1c1cf50036ba None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:50:57.131172 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e2958-d107-498e-ba57-068d067255fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:50:57.132135 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e2958-d107-498e-ba57-068d067255fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:57.132650 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e2958-d107-498e-ba57-068d067255fa admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:57.133157 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e2958-d107-498e-ba57-068d067255fa admin admin] PATH_INFO: `/v3/users/ce15e9a8d1e245aa94dbe479c76cc2d9` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:57.135324 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e2958-d107-498e-ba57-068d067255fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce15e9a8d1e245aa94dbe479c76cc2d9)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:50:57.138910 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e2958-d107-498e-ba57-068d067255fa admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:50:57.205172 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-357e2958-d107-498e-ba57-068d067255fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce15e9a8d1e245aa94dbe479c76cc2d9'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:50:57.206285 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-357e2958-d107-498e-ba57-068d067255fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce15e9a8d1e245aa94dbe479c76cc2d9'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:50:57.220763 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-357e2958-d107-498e-ba57-068d067255fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce15e9a8d1e245aa94dbe479c76cc2d9'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:50:57.230129 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-357e2958-d107-498e-ba57-068d067255fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce15e9a8d1e245aa94dbe479c76cc2d9'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:50:57.231565 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-357e2958-d107-498e-ba57-068d067255fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce15e9a8d1e245aa94dbe479c76cc2d9'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:50:57.261683 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1169/2334] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:50:57 2022] DELETE /identity/v3/users/ce15e9a8d1e245aa94dbe479c76cc2d9 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:50:57.262578 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6d124-87b6-4769-932a-9eda63a0d9bf None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:57.263050 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6d124-87b6-4769-932a-9eda63a0d9bf None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:57.263252 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6d124-87b6-4769-932a-9eda63a0d9bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:57.282740 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-3cb6d124-87b6-4769-932a-9eda63a0d9bf None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:50:57.283944 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-3cb6d124-87b6-4769-932a-9eda63a0d9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:50:57.346437 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1166/2335] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:50:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:50:57.356674 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e2958-d107-498e-ba57-068d067255fa admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:50:57.358122 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-357e2958-d107-498e-ba57-068d067255fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:50:57.417143 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e2958-d107-498e-ba57-068d067255fa admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:50:57.420829 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef9b2de-b982-480e-ac43-ca4d92912c94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:50:57.421426 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef9b2de-b982-480e-ac43-ca4d92912c94 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:57.421700 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef9b2de-b982-480e-ac43-ca4d92912c94 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:57.421971 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef9b2de-b982-480e-ac43-ca4d92912c94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:57.430985 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef9b2de-b982-480e-ac43-ca4d92912c94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:50:57.432729 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef9b2de-b982-480e-ac43-ca4d92912c94 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:50:57.443334 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1170/2336] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:50:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:50:57.908383 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6d124-87b6-4769-932a-9eda63a0d9bf None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:50:57.909780 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-3cb6d124-87b6-4769-932a-9eda63a0d9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:50:57.921934 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6d124-87b6-4769-932a-9eda63a0d9bf None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:50:57.926565 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b0a135-f51a-47b9-b680-5c23e32739c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:50:57.927354 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0a135-f51a-47b9-b680-5c23e32739c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:57.927645 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0a135-f51a-47b9-b680-5c23e32739c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:57.927919 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0a135-f51a-47b9-b680-5c23e32739c2 admin admin] PATH_INFO: `/v3/projects/0ad5c7721e2b44b7bc33221137152d56` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:57.929066 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b0a135-f51a-47b9-b680-5c23e32739c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ad5c7721e2b44b7bc33221137152d56)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:50:57.930973 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b0a135-f51a-47b9-b680-5c23e32739c2 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:50:57.993655 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-a4b0a135-f51a-47b9-b680-5c23e32739c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ad5c7721e2b44b7bc33221137152d56', 'request_id': 'req-a4b0a135-f51a-47b9-b680-5c23e32739c2'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:50:57.999444 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-a4b0a135-f51a-47b9-b680-5c23e32739c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ad5c7721e2b44b7bc33221137152d56', 'request_id': 'req-a4b0a135-f51a-47b9-b680-5c23e32739c2'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:50:58.007104 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-a4b0a135-f51a-47b9-b680-5c23e32739c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ad5c7721e2b44b7bc33221137152d56', 'request_id': 'req-a4b0a135-f51a-47b9-b680-5c23e32739c2'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:50:58.019464 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-a4b0a135-f51a-47b9-b680-5c23e32739c2 admin admin] The token cache is being invalidate because project 0ad5c7721e2b44b7bc33221137152d56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:50:58.019756 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-a4b0a135-f51a-47b9-b680-5c23e32739c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:50:58.020451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-a4b0a135-f51a-47b9-b680-5c23e32739c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:50:58.022063 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1167/2337] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:50:57 2022] DELETE /identity/v3/projects/0ad5c7721e2b44b7bc33221137152d56 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:50:58.026940 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c22532-346a-4284-99e9-34797e2a79b6 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:58.027303 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c22532-346a-4284-99e9-34797e2a79b6 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:58.027601 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c22532-346a-4284-99e9-34797e2a79b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:58.047789 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-a2c22532-346a-4284-99e9-34797e2a79b6 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:50:58.049029 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-a2c22532-346a-4284-99e9-34797e2a79b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:50:58.108687 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1171/2338] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:50:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:50:58.116647 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30170f00-9e77-478c-a075-2f2f9c9a39e2 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:58.116924 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30170f00-9e77-478c-a075-2f2f9c9a39e2 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:58.117192 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30170f00-9e77-478c-a075-2f2f9c9a39e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:58.136485 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-30170f00-9e77-478c-a075-2f2f9c9a39e2 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:50:58.138195 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-30170f00-9e77-478c-a075-2f2f9c9a39e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:50:58.146147 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1168/2339] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:50:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:50:58.150174 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c22532-346a-4284-99e9-34797e2a79b6 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:50:58.161086 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c22532-346a-4284-99e9-34797e2a79b6 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:50:58.166553 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b2054-46c8-431e-b577-7b3c42109e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:50:58.167321 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b2054-46c8-431e-b577-7b3c42109e98 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:58.167695 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b2054-46c8-431e-b577-7b3c42109e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:58.168025 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b2054-46c8-431e-b577-7b3c42109e98 admin admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:58.168744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588b2054-46c8-431e-b577-7b3c42109e98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:50:58.169230 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588b2054-46c8-431e-b577-7b3c42109e98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:50:58.170900 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588b2054-46c8-431e-b577-7b3c42109e98 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:50:58.184000 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1172/2340] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:50:58.187869 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30170f00-9e77-478c-a075-2f2f9c9a39e2 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:50:58.197107 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30170f00-9e77-478c-a075-2f2f9c9a39e2 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:50:58.202254 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd2dd6b-fcee-4305-ae9f-1d3dd717f628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:50:58.202846 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2dd6b-fcee-4305-ae9f-1d3dd717f628 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:58.203120 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2dd6b-fcee-4305-ae9f-1d3dd717f628 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:58.203384 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2dd6b-fcee-4305-ae9f-1d3dd717f628 admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:58.204042 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd2dd6b-fcee-4305-ae9f-1d3dd717f628 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:50:58.205956 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd2dd6b-fcee-4305-ae9f-1d3dd717f628 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:50:58.234745 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1169/2341] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:50:58 2022] POST /identity/v3/projects => generated 405 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:50:58.238947 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b2054-46c8-431e-b577-7b3c42109e98 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:50:58.245707 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:50:58.251008 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b2054-46c8-431e-b577-7b3c42109e98 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:50:58.315043 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a982dc-211f-43cf-96c3-7b5f0147c654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:50:58.315655 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a982dc-211f-43cf-96c3-7b5f0147c654 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:58.315932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a982dc-211f-43cf-96c3-7b5f0147c654 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:58.316229 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a982dc-211f-43cf-96c3-7b5f0147c654 admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:58.316911 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a982dc-211f-43cf-96c3-7b5f0147c654 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:50:58.318882 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a982dc-211f-43cf-96c3-7b5f0147c654 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:50:58.342942 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1173/2342] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:50:58 2022] POST /identity/v3/users => generated 303 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:50:58.347015 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd2dd6b-fcee-4305-ae9f-1d3dd717f628 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:50:58.357515 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd2dd6b-fcee-4305-ae9f-1d3dd717f628 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:50:58.362329 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584ef06-e501-49d4-bd03-decdfd339468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:50:58.362956 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584ef06-e501-49d4-bd03-decdfd339468 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:58.363229 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584ef06-e501-49d4-bd03-decdfd339468 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:58.363501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584ef06-e501-49d4-bd03-decdfd339468 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:58.364178 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8584ef06-e501-49d4-bd03-decdfd339468 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:50:58.365785 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8584ef06-e501-49d4-bd03-decdfd339468 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:50:58.371044 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:50:58.379888 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1170/2343] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 23:50:58.381051 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a982dc-211f-43cf-96c3-7b5f0147c654 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:50:58.393503 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a982dc-211f-43cf-96c3-7b5f0147c654 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:50:58.397186 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e57ebfe-d4d2-4aaf-b32e-29898385fe99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:50:58.397800 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57ebfe-d4d2-4aaf-b32e-29898385fe99 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:58.398073 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57ebfe-d4d2-4aaf-b32e-29898385fe99 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:58.398362 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57ebfe-d4d2-4aaf-b32e-29898385fe99 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:58.399010 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e57ebfe-d4d2-4aaf-b32e-29898385fe99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:50:58.400637 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e57ebfe-d4d2-4aaf-b32e-29898385fe99 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:50:58.411662 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1174/2344] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:50:58.416357 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584ef06-e501-49d4-bd03-decdfd339468 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:50:58.426387 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584ef06-e501-49d4-bd03-decdfd339468 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:50:58.431132 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d64bba9-9ff7-44c6-a63f-3bdbb418eb57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:50:58.431780 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64bba9-9ff7-44c6-a63f-3bdbb418eb57 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:58.432089 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64bba9-9ff7-44c6-a63f-3bdbb418eb57 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:58.432357 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64bba9-9ff7-44c6-a63f-3bdbb418eb57 admin admin] PATH_INFO: `/v3/projects/8da4fc3935874d3699791b764a5eb7c2/users/daef2ce8aae64c4487c7f5fdac3972ea/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:58.449379 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d64bba9-9ff7-44c6-a63f-3bdbb418eb57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8da4fc3935874d3699791b764a5eb7c2, user_id=daef2ce8aae64c4487c7f5fdac3972ea, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:50:58.452862 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d64bba9-9ff7-44c6-a63f-3bdbb418eb57 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:50:58.467255 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1171/2345] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:50:58 2022] PUT /identity/v3/projects/8da4fc3935874d3699791b764a5eb7c2/users/daef2ce8aae64c4487c7f5fdac3972ea/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:50:58.472249 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d845fe5-4778-4c4f-86a7-aac9716b3c24 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:58.472532 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d845fe5-4778-4c4f-86a7-aac9716b3c24 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:58.472801 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d845fe5-4778-4c4f-86a7-aac9716b3c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:58.494487 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-4d845fe5-4778-4c4f-86a7-aac9716b3c24 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:50:58.495688 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-4d845fe5-4778-4c4f-86a7-aac9716b3c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:50:58.569322 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1175/2346] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:50:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:50:58.582300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d64bba9-9ff7-44c6-a63f-3bdbb418eb57 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:50:58.584071 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-5d64bba9-9ff7-44c6-a63f-3bdbb418eb57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:50:58.641237 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d64bba9-9ff7-44c6-a63f-3bdbb418eb57 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:50:58.644899 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b0ab5-06d5-4447-a636-085d1a6d5395 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:50:58.645508 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b0ab5-06d5-4447-a636-085d1a6d5395 service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:58.645785 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b0ab5-06d5-4447-a636-085d1a6d5395 service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:58.646060 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b0ab5-06d5-4447-a636-085d1a6d5395 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:58.655544 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5b0ab5-06d5-4447-a636-085d1a6d5395 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:50:58.657049 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5b0ab5-06d5-4447-a636-085d1a6d5395 service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:50:58.666844 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1172/2347] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:50:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:50:59.468474 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c17af6-786c-4192-b05a-c4c3696becdd None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:59.468977 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c17af6-786c-4192-b05a-c4c3696becdd None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:59.469153 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c17af6-786c-4192-b05a-c4c3696becdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:59.504226 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-f9c17af6-786c-4192-b05a-c4c3696becdd None None] MFA Rules not processed for user `daef2ce8aae64c4487c7f5fdac3972ea`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:50:59.505963 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-f9c17af6-786c-4192-b05a-c4c3696becdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:50:59.599172 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1176/2348] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:50:59 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:50:59.600758 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b0ab5-06d5-4447-a636-085d1a6d5395 service neutron] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:50:59.615130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b0ab5-06d5-4447-a636-085d1a6d5395 service neutron] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:50:59.620394 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a8ccb4-edc2-4be1-8318-ebf3ae5787e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:50:59.621242 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8ccb4-edc2-4be1-8318-ebf3ae5787e2 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:59.622267 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8ccb4-edc2-4be1-8318-ebf3ae5787e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:59.622700 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8ccb4-edc2-4be1-8318-ebf3ae5787e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:59.623515 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a8ccb4-edc2-4be1-8318-ebf3ae5787e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:50:59.625723 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a8ccb4-edc2-4be1-8318-ebf3ae5787e2 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:50:59.657594 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1173/2349] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:50:59 2022] POST /identity/v3/projects => generated 405 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:50:59.662813 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c17af6-786c-4192-b05a-c4c3696becdd None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:50:59.677332 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c17af6-786c-4192-b05a-c4c3696becdd None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:50:59.689486 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:50:59.761072 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a98eb1-119d-4ddb-af64-5b5488a72b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:50:59.761838 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a98eb1-119d-4ddb-af64-5b5488a72b18 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:59.762149 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a98eb1-119d-4ddb-af64-5b5488a72b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:59.762419 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a98eb1-119d-4ddb-af64-5b5488a72b18 admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:59.763046 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a98eb1-119d-4ddb-af64-5b5488a72b18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:50:59.765011 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a98eb1-119d-4ddb-af64-5b5488a72b18 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:50:59.792069 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1177/2350] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:50:59 2022] POST /identity/v3/users => generated 309 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:50:59.793126 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a8ccb4-edc2-4be1-8318-ebf3ae5787e2 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:50:59.805568 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a8ccb4-edc2-4be1-8318-ebf3ae5787e2 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:50:59.811295 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9927917-9a49-4124-a06a-fa7e5051f7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:50:59.812016 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9927917-9a49-4124-a06a-fa7e5051f7f7 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:59.812336 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9927917-9a49-4124-a06a-fa7e5051f7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:59.812609 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9927917-9a49-4124-a06a-fa7e5051f7f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:59.813263 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9927917-9a49-4124-a06a-fa7e5051f7f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:50:59.814875 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9927917-9a49-4124-a06a-fa7e5051f7f7 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:50:59.825957 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1174/2351] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 23:50:59.830387 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a98eb1-119d-4ddb-af64-5b5488a72b18 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:50:59.841664 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a98eb1-119d-4ddb-af64-5b5488a72b18 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:50:59.847341 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3420f5-4fcd-4898-acc4-96471882d4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:50:59.848094 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3420f5-4fcd-4898-acc4-96471882d4ef admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:59.848454 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3420f5-4fcd-4898-acc4-96471882d4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:59.848788 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3420f5-4fcd-4898-acc4-96471882d4ef admin admin] PATH_INFO: `/v3/projects/60974fd9751041e58be86e8312e69238/users/7f5291bb43ca4138919f5b64cd49c05d/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:59.865279 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3420f5-4fcd-4898-acc4-96471882d4ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=60974fd9751041e58be86e8312e69238, user_id=7f5291bb43ca4138919f5b64cd49c05d, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:50:59.868059 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3420f5-4fcd-4898-acc4-96471882d4ef admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:50:59.880726 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1178/2352] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:50:59 2022] PUT /identity/v3/projects/60974fd9751041e58be86e8312e69238/users/7f5291bb43ca4138919f5b64cd49c05d/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:50:59.884726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9927917-9a49-4124-a06a-fa7e5051f7f7 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:50:59.894784 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9927917-9a49-4124-a06a-fa7e5051f7f7 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:50:59.952179 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b6de31-0a35-41c2-8333-41ff787486f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:50:59.952783 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6de31-0a35-41c2-8333-41ff787486f5 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:59.953061 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6de31-0a35-41c2-8333-41ff787486f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:59.953328 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6de31-0a35-41c2-8333-41ff787486f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:59.953963 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b6de31-0a35-41c2-8333-41ff787486f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:50:59.955551 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b6de31-0a35-41c2-8333-41ff787486f5 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:50:59.965972 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1175/2353] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 23:50:59.970464 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3420f5-4fcd-4898-acc4-96471882d4ef admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:50:59.980779 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3420f5-4fcd-4898-acc4-96471882d4ef admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:50:59.986096 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b0afa6-e223-44e9-aea4-6c7c1310979e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:50:59.987170 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0afa6-e223-44e9-aea4-6c7c1310979e admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:50:59.987170 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0afa6-e223-44e9-aea4-6c7c1310979e admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:50:59.987383 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0afa6-e223-44e9-aea4-6c7c1310979e admin admin] PATH_INFO: `/v3/projects/60974fd9751041e58be86e8312e69238/users/7f5291bb43ca4138919f5b64cd49c05d/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:50:59.989139 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b0afa6-e223-44e9-aea4-6c7c1310979e admin admin] RBAC: Authorizing `identity:create_grant(project_id=60974fd9751041e58be86e8312e69238, user_id=7f5291bb43ca4138919f5b64cd49c05d, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:50:59.991803 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b0afa6-e223-44e9-aea4-6c7c1310979e admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:00.002810 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1179/2354] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:50:59 2022] PUT /identity/v3/projects/60974fd9751041e58be86e8312e69238/users/7f5291bb43ca4138919f5b64cd49c05d/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:51:00.663890 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f692b87-d4a0-409a-aa98-926173ab641f None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:00.664545 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f692b87-d4a0-409a-aa98-926173ab641f None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:00.664545 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f692b87-d4a0-409a-aa98-926173ab641f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:00.693129 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-9f692b87-d4a0-409a-aa98-926173ab641f None None] MFA Rules not processed for user `7f5291bb43ca4138919f5b64cd49c05d`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:51:00.694335 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-9f692b87-d4a0-409a-aa98-926173ab641f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:00.727003 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:51:00.778126 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1176/2355] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:51:00 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:51:01.399425 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3631d2-09b5-48f9-98ad-5a58709f605c None None] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:01.399899 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3631d2-09b5-48f9-98ad-5a58709f605c None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:01.400020 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3631d2-09b5-48f9-98ad-5a58709f605c None None] PATH_INFO: `/` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:01.401361 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1180/2356] 10.222.0.26 () {54 vars in 921 bytes} [Wed Aug 17 23:51:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 23:51:01.411772 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ed5c9-2b89-4085-bc40-1d3ce3ebb5a5 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:01.412355 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ed5c9-2b89-4085-bc40-1d3ce3ebb5a5 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:01.412632 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ed5c9-2b89-4085-bc40-1d3ce3ebb5a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:01.432578 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-de2ed5c9-2b89-4085-bc40-1d3ce3ebb5a5 None None] MFA Rules not processed for user `1fc57d911a56423e96c736bc96270ba8`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:51:01.433790 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-de2ed5c9-2b89-4085-bc40-1d3ce3ebb5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:01.511598 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1177/2357] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Aug 17 23:51:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:51:01.523433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3631d2-09b5-48f9-98ad-5a58709f605c None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:01.525974 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-8f3631d2-09b5-48f9-98ad-5a58709f605c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:01.544606 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3631d2-09b5-48f9-98ad-5a58709f605c None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:01.549568 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918b79c-c6b7-4fa2-9a43-dc5819eceec2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc57d911a56423e96c736bc96270ba8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:01.550172 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918b79c-c6b7-4fa2-9a43-dc5819eceec2 service placement] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:01.550448 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918b79c-c6b7-4fa2-9a43-dc5819eceec2 service placement] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:01.550716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918b79c-c6b7-4fa2-9a43-dc5819eceec2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:01.560829 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2918b79c-c6b7-4fa2-9a43-dc5819eceec2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:01.562450 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2918b79c-c6b7-4fa2-9a43-dc5819eceec2 service placement] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:01.573496 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1181/2358] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:51:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:51:02.518648 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ed5c9-2b89-4085-bc40-1d3ce3ebb5a5 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:02.600827 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ed5c9-2b89-4085-bc40-1d3ce3ebb5a5 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:02.605572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471bc9c5-6641-49e6-8e3a-d73fa4b120d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:02.606732 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471bc9c5-6641-49e6-8e3a-d73fa4b120d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:02.607365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471bc9c5-6641-49e6-8e3a-d73fa4b120d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:02.607365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471bc9c5-6641-49e6-8e3a-d73fa4b120d0 admin admin] PATH_INFO: `/v3/users/daef2ce8aae64c4487c7f5fdac3972ea` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:02.608500 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471bc9c5-6641-49e6-8e3a-d73fa4b120d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=daef2ce8aae64c4487c7f5fdac3972ea)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:02.610755 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471bc9c5-6641-49e6-8e3a-d73fa4b120d0 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:02.676599 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-471bc9c5-6641-49e6-8e3a-d73fa4b120d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'daef2ce8aae64c4487c7f5fdac3972ea'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:02.677346 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-471bc9c5-6641-49e6-8e3a-d73fa4b120d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'daef2ce8aae64c4487c7f5fdac3972ea'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:02.677987 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-471bc9c5-6641-49e6-8e3a-d73fa4b120d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'daef2ce8aae64c4487c7f5fdac3972ea'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:02.685643 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-471bc9c5-6641-49e6-8e3a-d73fa4b120d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'daef2ce8aae64c4487c7f5fdac3972ea'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:02.694763 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-471bc9c5-6641-49e6-8e3a-d73fa4b120d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'daef2ce8aae64c4487c7f5fdac3972ea'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:02.697470 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:51:02.715457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1178/2359] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:51:02 2022] DELETE /identity/v3/users/daef2ce8aae64c4487c7f5fdac3972ea => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:51:02.902090 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918b79c-c6b7-4fa2-9a43-dc5819eceec2 service placement] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:02.921507 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918b79c-c6b7-4fa2-9a43-dc5819eceec2 service placement] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:02.985458 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b728f2a1-3fd2-4c1c-907b-2df00ac7056d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:02.986100 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728f2a1-3fd2-4c1c-907b-2df00ac7056d admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:02.986380 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728f2a1-3fd2-4c1c-907b-2df00ac7056d admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:02.986651 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728f2a1-3fd2-4c1c-907b-2df00ac7056d admin admin] PATH_INFO: `/v3/projects/8da4fc3935874d3699791b764a5eb7c2` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:02.987916 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b728f2a1-3fd2-4c1c-907b-2df00ac7056d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8da4fc3935874d3699791b764a5eb7c2)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:02.989913 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b728f2a1-3fd2-4c1c-907b-2df00ac7056d admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:03.037673 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-b728f2a1-3fd2-4c1c-907b-2df00ac7056d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8da4fc3935874d3699791b764a5eb7c2', 'request_id': 'req-b728f2a1-3fd2-4c1c-907b-2df00ac7056d'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:03.041495 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:51:03.044570 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-b728f2a1-3fd2-4c1c-907b-2df00ac7056d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8da4fc3935874d3699791b764a5eb7c2', 'request_id': 'req-b728f2a1-3fd2-4c1c-907b-2df00ac7056d'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:03.053649 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-b728f2a1-3fd2-4c1c-907b-2df00ac7056d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8da4fc3935874d3699791b764a5eb7c2', 'request_id': 'req-b728f2a1-3fd2-4c1c-907b-2df00ac7056d'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:03.065476 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-b728f2a1-3fd2-4c1c-907b-2df00ac7056d admin admin] The token cache is being invalidate because project 8da4fc3935874d3699791b764a5eb7c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:51:03.065765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-b728f2a1-3fd2-4c1c-907b-2df00ac7056d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:03.066406 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-b728f2a1-3fd2-4c1c-907b-2df00ac7056d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:03.068196 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1182/2360] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:51:02 2022] DELETE /identity/v3/projects/8da4fc3935874d3699791b764a5eb7c2 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:51:03.072228 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471bc9c5-6641-49e6-8e3a-d73fa4b120d0 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:03.073724 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-471bc9c5-6641-49e6-8e3a-d73fa4b120d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:03.086735 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471bc9c5-6641-49e6-8e3a-d73fa4b120d0 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:03.148115 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7a61b-0606-4bd9-996b-91ad4d82e2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:03.148766 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7a61b-0606-4bd9-996b-91ad4d82e2ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:03.149045 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7a61b-0606-4bd9-996b-91ad4d82e2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:03.149320 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7a61b-0606-4bd9-996b-91ad4d82e2ad admin admin] PATH_INFO: `/v3/users/7f5291bb43ca4138919f5b64cd49c05d` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:03.150477 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e7a61b-0606-4bd9-996b-91ad4d82e2ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f5291bb43ca4138919f5b64cd49c05d)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:03.152586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e7a61b-0606-4bd9-996b-91ad4d82e2ad admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:03.201519 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-55e7a61b-0606-4bd9-996b-91ad4d82e2ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f5291bb43ca4138919f5b64cd49c05d'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:03.202185 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-55e7a61b-0606-4bd9-996b-91ad4d82e2ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f5291bb43ca4138919f5b64cd49c05d'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:03.202828 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-55e7a61b-0606-4bd9-996b-91ad4d82e2ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f5291bb43ca4138919f5b64cd49c05d'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:03.211780 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-55e7a61b-0606-4bd9-996b-91ad4d82e2ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f5291bb43ca4138919f5b64cd49c05d'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:03.220208 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-55e7a61b-0606-4bd9-996b-91ad4d82e2ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f5291bb43ca4138919f5b64cd49c05d'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:03.238508 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1179/2361] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:51:03 2022] DELETE /identity/v3/users/7f5291bb43ca4138919f5b64cd49c05d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:51:03.513908 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b728f2a1-3fd2-4c1c-907b-2df00ac7056d admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:03.515642 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-b728f2a1-3fd2-4c1c-907b-2df00ac7056d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:03.531275 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b728f2a1-3fd2-4c1c-907b-2df00ac7056d admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:03.600977 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1daa39-372b-4a2d-959a-7f841f4f1a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:03.601565 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1daa39-372b-4a2d-959a-7f841f4f1a9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:03.601982 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1daa39-372b-4a2d-959a-7f841f4f1a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:03.602261 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1daa39-372b-4a2d-959a-7f841f4f1a9b admin admin] PATH_INFO: `/v3/projects/60974fd9751041e58be86e8312e69238` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:03.603551 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1daa39-372b-4a2d-959a-7f841f4f1a9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=60974fd9751041e58be86e8312e69238)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:03.605613 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1daa39-372b-4a2d-959a-7f841f4f1a9b admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:03.652934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-3d1daa39-372b-4a2d-959a-7f841f4f1a9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60974fd9751041e58be86e8312e69238', 'request_id': 'req-3d1daa39-372b-4a2d-959a-7f841f4f1a9b'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:03.658424 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-3d1daa39-372b-4a2d-959a-7f841f4f1a9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60974fd9751041e58be86e8312e69238', 'request_id': 'req-3d1daa39-372b-4a2d-959a-7f841f4f1a9b'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:03.667569 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-3d1daa39-372b-4a2d-959a-7f841f4f1a9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60974fd9751041e58be86e8312e69238', 'request_id': 'req-3d1daa39-372b-4a2d-959a-7f841f4f1a9b'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:03.680571 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-3d1daa39-372b-4a2d-959a-7f841f4f1a9b admin admin] The token cache is being invalidate because project 60974fd9751041e58be86e8312e69238 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:51:03.680861 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-3d1daa39-372b-4a2d-959a-7f841f4f1a9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:03.681563 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-3d1daa39-372b-4a2d-959a-7f841f4f1a9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:03.683296 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1183/2362] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:51:03 2022] DELETE /identity/v3/projects/60974fd9751041e58be86e8312e69238 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:51:03.688730 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc40e3f-a879-4048-86ef-01da0c742172 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:03.689194 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc40e3f-a879-4048-86ef-01da0c742172 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:03.689289 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc40e3f-a879-4048-86ef-01da0c742172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:03.716425 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-edc40e3f-a879-4048-86ef-01da0c742172 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:51:03.717652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-edc40e3f-a879-4048-86ef-01da0c742172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:03.778445 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1180/2363] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:51:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:51:03.786483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e9f90-096f-479e-bb8f-3f0644b95e6d None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:03.786794 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e9f90-096f-479e-bb8f-3f0644b95e6d None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:03.787351 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e9f90-096f-479e-bb8f-3f0644b95e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:03.807201 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-834e9f90-096f-479e-bb8f-3f0644b95e6d None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:51:03.808430 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-834e9f90-096f-479e-bb8f-3f0644b95e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:03.816910 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1184/2364] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:51:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:51:03.821258 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc40e3f-a879-4048-86ef-01da0c742172 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:03.826632 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:51:03.833077 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc40e3f-a879-4048-86ef-01da0c742172 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:03.838573 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cefefcb-f4ba-410f-8b5b-df61579599de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:03.839170 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefefcb-f4ba-410f-8b5b-df61579599de admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:03.839450 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefefcb-f4ba-410f-8b5b-df61579599de admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:03.839719 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefefcb-f4ba-410f-8b5b-df61579599de admin admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:03.840440 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cefefcb-f4ba-410f-8b5b-df61579599de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:51:03.841270 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cefefcb-f4ba-410f-8b5b-df61579599de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:03.842373 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cefefcb-f4ba-410f-8b5b-df61579599de admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:03.855327 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1181/2365] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:51:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:51:03.859848 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834e9f90-096f-479e-bb8f-3f0644b95e6d None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:03.869304 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834e9f90-096f-479e-bb8f-3f0644b95e6d None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:03.874525 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29917cb8-3efc-49c3-a006-81de3a9a5a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:03.875192 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29917cb8-3efc-49c3-a006-81de3a9a5a06 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:03.875555 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29917cb8-3efc-49c3-a006-81de3a9a5a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:03.875889 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29917cb8-3efc-49c3-a006-81de3a9a5a06 admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:03.876603 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29917cb8-3efc-49c3-a006-81de3a9a5a06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:03.878601 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29917cb8-3efc-49c3-a006-81de3a9a5a06 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:03.910466 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1185/2366] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:51:03 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:51:03.911656 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cefefcb-f4ba-410f-8b5b-df61579599de admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:03.922663 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cefefcb-f4ba-410f-8b5b-df61579599de admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:03.982053 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcdf3b5-4b59-475d-8663-a2fa0710e31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:03.982675 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcdf3b5-4b59-475d-8663-a2fa0710e31e admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:03.982956 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcdf3b5-4b59-475d-8663-a2fa0710e31e admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:03.983231 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcdf3b5-4b59-475d-8663-a2fa0710e31e admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:03.983902 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcdf3b5-4b59-475d-8663-a2fa0710e31e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:03.985866 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcdf3b5-4b59-475d-8663-a2fa0710e31e admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:04.006718 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1182/2367] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:51:03 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:51:04.010833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29917cb8-3efc-49c3-a006-81de3a9a5a06 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:04.021048 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29917cb8-3efc-49c3-a006-81de3a9a5a06 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:04.026002 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e4fd1-caee-489f-84ca-29b5457a15d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:04.026597 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e4fd1-caee-489f-84ca-29b5457a15d3 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:04.026874 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e4fd1-caee-489f-84ca-29b5457a15d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:04.027143 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e4fd1-caee-489f-84ca-29b5457a15d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:04.027829 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017e4fd1-caee-489f-84ca-29b5457a15d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:04.029467 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017e4fd1-caee-489f-84ca-29b5457a15d3 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:04.039944 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1186/2368] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:51:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:51:04.044470 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcdf3b5-4b59-475d-8663-a2fa0710e31e admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:04.054711 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcdf3b5-4b59-475d-8663-a2fa0710e31e admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:04.059853 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08adddb-2d79-4a7a-8158-bc14d03f450b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:04.060487 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08adddb-2d79-4a7a-8158-bc14d03f450b admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:04.060810 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08adddb-2d79-4a7a-8158-bc14d03f450b admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:04.061086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08adddb-2d79-4a7a-8158-bc14d03f450b admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:04.061726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08adddb-2d79-4a7a-8158-bc14d03f450b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:04.063304 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08adddb-2d79-4a7a-8158-bc14d03f450b admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:04.074456 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1183/2369] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:51:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:51:04.078782 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e4fd1-caee-489f-84ca-29b5457a15d3 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:04.091238 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e4fd1-caee-489f-84ca-29b5457a15d3 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:04.096379 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f465fc-5c9a-448a-8a00-43dc3e921fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:04.097031 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f465fc-5c9a-448a-8a00-43dc3e921fbc admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:04.097312 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f465fc-5c9a-448a-8a00-43dc3e921fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:04.097631 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f465fc-5c9a-448a-8a00-43dc3e921fbc admin admin] PATH_INFO: `/v3/projects/4687fad94ce14c28bfcaac664f3c88e2/users/f54e3d7a3a5148b68aba0ef67da48bdb/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:04.117189 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f465fc-5c9a-448a-8a00-43dc3e921fbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4687fad94ce14c28bfcaac664f3c88e2, user_id=f54e3d7a3a5148b68aba0ef67da48bdb, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:04.120601 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f465fc-5c9a-448a-8a00-43dc3e921fbc admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:04.136927 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1187/2370] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:51:04 2022] PUT /identity/v3/projects/4687fad94ce14c28bfcaac664f3c88e2/users/f54e3d7a3a5148b68aba0ef67da48bdb/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:51:04.148591 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba854efa-58a4-4d04-b623-b430dde389d4 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:04.148878 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba854efa-58a4-4d04-b623-b430dde389d4 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:04.149179 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba854efa-58a4-4d04-b623-b430dde389d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:04.198173 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-ba854efa-58a4-4d04-b623-b430dde389d4 None None] MFA Rules not processed for user `f54e3d7a3a5148b68aba0ef67da48bdb`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:51:04.199435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-ba854efa-58a4-4d04-b623-b430dde389d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:04.280277 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1184/2371] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:51:04 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:51:04.284703 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f465fc-5c9a-448a-8a00-43dc3e921fbc admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:04.297433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:51:04.400507 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f465fc-5c9a-448a-8a00-43dc3e921fbc admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:04.406415 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5adc174-4655-45b1-ad85-a4abdb835986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:04.407032 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adc174-4655-45b1-ad85-a4abdb835986 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:04.407308 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adc174-4655-45b1-ad85-a4abdb835986 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:04.407574 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adc174-4655-45b1-ad85-a4abdb835986 admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:04.408282 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5adc174-4655-45b1-ad85-a4abdb835986 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:04.410216 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5adc174-4655-45b1-ad85-a4abdb835986 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:04.454236 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1188/2372] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:51:04 2022] POST /identity/v3/projects => generated 387 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:51:04.456054 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba854efa-58a4-4d04-b623-b430dde389d4 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:04.470240 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba854efa-58a4-4d04-b623-b430dde389d4 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:04.540721 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:51:04.549293 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b630bb-1fa9-4625-af74-0da1fbd25589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:04.549915 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b630bb-1fa9-4625-af74-0da1fbd25589 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:04.550517 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b630bb-1fa9-4625-af74-0da1fbd25589 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:04.552007 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b630bb-1fa9-4625-af74-0da1fbd25589 admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:04.552751 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b630bb-1fa9-4625-af74-0da1fbd25589 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:04.555710 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b630bb-1fa9-4625-af74-0da1fbd25589 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:04.579998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1185/2373] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:51:04 2022] POST /identity/v3/users => generated 300 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:51:04.582493 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5adc174-4655-45b1-ad85-a4abdb835986 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:04.601400 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5adc174-4655-45b1-ad85-a4abdb835986 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:04.607855 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a91a5-931a-4321-8e5f-894fb5f85ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:04.608705 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a91a5-931a-4321-8e5f-894fb5f85ddc admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:04.609271 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a91a5-931a-4321-8e5f-894fb5f85ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:04.609271 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a91a5-931a-4321-8e5f-894fb5f85ddc admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:04.609914 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83a91a5-931a-4321-8e5f-894fb5f85ddc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:04.611632 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83a91a5-931a-4321-8e5f-894fb5f85ddc admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:04.624163 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1189/2374] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:51:04 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:51:04.634585 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b630bb-1fa9-4625-af74-0da1fbd25589 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:04.645221 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b630bb-1fa9-4625-af74-0da1fbd25589 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:04.650083 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd0f85-d889-48b3-b79f-fb1c9d1d9e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:04.650739 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd0f85-d889-48b3-b79f-fb1c9d1d9e36 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:04.651015 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd0f85-d889-48b3-b79f-fb1c9d1d9e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:04.651287 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd0f85-d889-48b3-b79f-fb1c9d1d9e36 admin admin] PATH_INFO: `/v3/projects/a44c5d1aa5c24288a92072c5181d9166/users/f54d6dce161e45ff8f4fcc57c6b9b8e6/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:04.667511 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dd0f85-d889-48b3-b79f-fb1c9d1d9e36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a44c5d1aa5c24288a92072c5181d9166, user_id=f54d6dce161e45ff8f4fcc57c6b9b8e6, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:04.670408 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dd0f85-d889-48b3-b79f-fb1c9d1d9e36 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:04.689063 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1186/2375] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:51:04 2022] PUT /identity/v3/projects/a44c5d1aa5c24288a92072c5181d9166/users/f54d6dce161e45ff8f4fcc57c6b9b8e6/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:51:04.698962 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a91a5-931a-4321-8e5f-894fb5f85ddc admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:04.709682 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a91a5-931a-4321-8e5f-894fb5f85ddc admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:04.775810 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad8ff70-64e1-4959-bd0f-2348610f129b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:04.776459 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8ff70-64e1-4959-bd0f-2348610f129b admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:04.776737 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8ff70-64e1-4959-bd0f-2348610f129b admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:04.777007 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8ff70-64e1-4959-bd0f-2348610f129b admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:04.778270 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad8ff70-64e1-4959-bd0f-2348610f129b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:04.779913 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad8ff70-64e1-4959-bd0f-2348610f129b admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:04.790133 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1190/2376] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:51:04 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:51:04.794337 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd0f85-d889-48b3-b79f-fb1c9d1d9e36 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:04.804562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd0f85-d889-48b3-b79f-fb1c9d1d9e36 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:04.809401 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34dd301-3add-4490-b3ea-dc72552ea214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:04.810050 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34dd301-3add-4490-b3ea-dc72552ea214 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:04.810333 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34dd301-3add-4490-b3ea-dc72552ea214 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:04.810606 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34dd301-3add-4490-b3ea-dc72552ea214 admin admin] PATH_INFO: `/v3/projects/a44c5d1aa5c24288a92072c5181d9166/users/f54d6dce161e45ff8f4fcc57c6b9b8e6/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:04.812547 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34dd301-3add-4490-b3ea-dc72552ea214 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a44c5d1aa5c24288a92072c5181d9166, user_id=f54d6dce161e45ff8f4fcc57c6b9b8e6, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:04.815205 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34dd301-3add-4490-b3ea-dc72552ea214 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:04.826214 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1187/2377] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:51:04 2022] PUT /identity/v3/projects/a44c5d1aa5c24288a92072c5181d9166/users/f54d6dce161e45ff8f4fcc57c6b9b8e6/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:51:04.836567 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970a09e-19e6-426f-bbb8-644d4d1f2830 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:04.836855 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970a09e-19e6-426f-bbb8-644d4d1f2830 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:04.837131 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970a09e-19e6-426f-bbb8-644d4d1f2830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:04.869470 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-c970a09e-19e6-426f-bbb8-644d4d1f2830 None None] MFA Rules not processed for user `f54d6dce161e45ff8f4fcc57c6b9b8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:51:04.870755 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-c970a09e-19e6-426f-bbb8-644d4d1f2830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:04.960930 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1191/2378] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:51:04 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:51:04.973712 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34dd301-3add-4490-b3ea-dc72552ea214 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:04.975527 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-f34dd301-3add-4490-b3ea-dc72552ea214 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:05.060244 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34dd301-3add-4490-b3ea-dc72552ea214 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:05.065022 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c0ac0f-cd68-460a-b62b-eff700eb4d2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:05.065739 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0ac0f-cd68-460a-b62b-eff700eb4d2e service cinder] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:05.066136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0ac0f-cd68-460a-b62b-eff700eb4d2e service cinder] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:05.066505 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0ac0f-cd68-460a-b62b-eff700eb4d2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:05.078738 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c0ac0f-cd68-460a-b62b-eff700eb4d2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:05.080553 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c0ac0f-cd68-460a-b62b-eff700eb4d2e service cinder] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:05.091386 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1188/2379] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:51:04 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:51:05.161542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970a09e-19e6-426f-bbb8-644d4d1f2830 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:05.167533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:51:05.240599 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970a09e-19e6-426f-bbb8-644d4d1f2830 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:05.245420 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140580d-d43b-4a95-9cff-37ce4a1b1e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:05.246047 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140580d-d43b-4a95-9cff-37ce4a1b1e36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:05.246848 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140580d-d43b-4a95-9cff-37ce4a1b1e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:05.247118 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140580d-d43b-4a95-9cff-37ce4a1b1e36 admin admin] PATH_INFO: `/v3/users/f54e3d7a3a5148b68aba0ef67da48bdb` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:05.248428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1140580d-d43b-4a95-9cff-37ce4a1b1e36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f54e3d7a3a5148b68aba0ef67da48bdb)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:05.250535 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1140580d-d43b-4a95-9cff-37ce4a1b1e36 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:05.313114 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-1140580d-d43b-4a95-9cff-37ce4a1b1e36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f54e3d7a3a5148b68aba0ef67da48bdb'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:05.313842 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-1140580d-d43b-4a95-9cff-37ce4a1b1e36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f54e3d7a3a5148b68aba0ef67da48bdb'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:05.328667 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-1140580d-d43b-4a95-9cff-37ce4a1b1e36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f54e3d7a3a5148b68aba0ef67da48bdb'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:05.337542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-1140580d-d43b-4a95-9cff-37ce4a1b1e36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f54e3d7a3a5148b68aba0ef67da48bdb'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:05.338279 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-1140580d-d43b-4a95-9cff-37ce4a1b1e36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f54e3d7a3a5148b68aba0ef67da48bdb'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:05.356278 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1192/2380] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:51:05 2022] DELETE /identity/v3/users/f54e3d7a3a5148b68aba0ef67da48bdb => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:51:05.358061 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696caa34-5bc5-45a8-9e28-e1464254ef0d None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:05.359176 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696caa34-5bc5-45a8-9e28-e1464254ef0d None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:05.359520 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696caa34-5bc5-45a8-9e28-e1464254ef0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:05.381589 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-696caa34-5bc5-45a8-9e28-e1464254ef0d None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:51:05.383040 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-696caa34-5bc5-45a8-9e28-e1464254ef0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:05.494797 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1189/2381] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:51:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:51:05.505345 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140580d-d43b-4a95-9cff-37ce4a1b1e36 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:05.507297 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-1140580d-d43b-4a95-9cff-37ce4a1b1e36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:05.575810 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140580d-d43b-4a95-9cff-37ce4a1b1e36 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:05.579536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d0394-13f1-4410-b4ea-84fd75d0e62d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:05.580261 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d0394-13f1-4410-b4ea-84fd75d0e62d service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:05.580802 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d0394-13f1-4410-b4ea-84fd75d0e62d service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:05.581158 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d0394-13f1-4410-b4ea-84fd75d0e62d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:05.594946 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d0394-13f1-4410-b4ea-84fd75d0e62d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:05.596636 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d0394-13f1-4410-b4ea-84fd75d0e62d service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:05.607145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1193/2382] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:51:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:51:05.892706 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696caa34-5bc5-45a8-9e28-e1464254ef0d None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:05.894228 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-696caa34-5bc5-45a8-9e28-e1464254ef0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:05.908156 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696caa34-5bc5-45a8-9e28-e1464254ef0d None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:05.913665 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6871540d-62c3-4805-9ec9-c58868bb6960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:05.914690 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871540d-62c3-4805-9ec9-c58868bb6960 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:05.914690 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871540d-62c3-4805-9ec9-c58868bb6960 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:05.915097 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871540d-62c3-4805-9ec9-c58868bb6960 admin admin] PATH_INFO: `/v3/projects/4687fad94ce14c28bfcaac664f3c88e2` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:05.916377 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6871540d-62c3-4805-9ec9-c58868bb6960 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4687fad94ce14c28bfcaac664f3c88e2)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:05.919034 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6871540d-62c3-4805-9ec9-c58868bb6960 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:05.973476 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:51:05.975043 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-6871540d-62c3-4805-9ec9-c58868bb6960 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4687fad94ce14c28bfcaac664f3c88e2', 'request_id': 'req-6871540d-62c3-4805-9ec9-c58868bb6960'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:05.982813 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-6871540d-62c3-4805-9ec9-c58868bb6960 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4687fad94ce14c28bfcaac664f3c88e2', 'request_id': 'req-6871540d-62c3-4805-9ec9-c58868bb6960'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:05.990573 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-6871540d-62c3-4805-9ec9-c58868bb6960 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4687fad94ce14c28bfcaac664f3c88e2', 'request_id': 'req-6871540d-62c3-4805-9ec9-c58868bb6960'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:06.001565 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-6871540d-62c3-4805-9ec9-c58868bb6960 admin admin] The token cache is being invalidate because project 4687fad94ce14c28bfcaac664f3c88e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:51:06.001855 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-6871540d-62c3-4805-9ec9-c58868bb6960 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:06.002482 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-6871540d-62c3-4805-9ec9-c58868bb6960 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:06.003985 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1190/2383] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:51:05 2022] DELETE /identity/v3/projects/4687fad94ce14c28bfcaac664f3c88e2 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:51:06.008067 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d0394-13f1-4410-b4ea-84fd75d0e62d service neutron] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:06.017581 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d0394-13f1-4410-b4ea-84fd75d0e62d service neutron] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:06.041341 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:51:06.075677 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39afa1ae-d7ce-4848-aefc-10247356cae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:06.076345 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39afa1ae-d7ce-4848-aefc-10247356cae8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:06.076626 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39afa1ae-d7ce-4848-aefc-10247356cae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:06.077488 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39afa1ae-d7ce-4848-aefc-10247356cae8 admin admin] PATH_INFO: `/v3/users/f54d6dce161e45ff8f4fcc57c6b9b8e6` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:06.078622 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39afa1ae-d7ce-4848-aefc-10247356cae8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f54d6dce161e45ff8f4fcc57c6b9b8e6)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:06.080793 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39afa1ae-d7ce-4848-aefc-10247356cae8 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:06.124890 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-39afa1ae-d7ce-4848-aefc-10247356cae8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f54d6dce161e45ff8f4fcc57c6b9b8e6'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:06.125537 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-39afa1ae-d7ce-4848-aefc-10247356cae8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f54d6dce161e45ff8f4fcc57c6b9b8e6'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:06.137808 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-39afa1ae-d7ce-4848-aefc-10247356cae8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f54d6dce161e45ff8f4fcc57c6b9b8e6'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:06.144943 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-39afa1ae-d7ce-4848-aefc-10247356cae8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f54d6dce161e45ff8f4fcc57c6b9b8e6'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:06.145555 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-39afa1ae-d7ce-4848-aefc-10247356cae8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f54d6dce161e45ff8f4fcc57c6b9b8e6'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:06.158274 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1194/2384] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:51:06 2022] DELETE /identity/v3/users/f54d6dce161e45ff8f4fcc57c6b9b8e6 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:51:06.428340 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6871540d-62c3-4805-9ec9-c58868bb6960 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:06.429895 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-6871540d-62c3-4805-9ec9-c58868bb6960 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:06.443749 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6871540d-62c3-4805-9ec9-c58868bb6960 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:06.504249 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4750b8-4ce9-4541-a733-d3324b89e198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:06.504870 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4750b8-4ce9-4541-a733-d3324b89e198 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:06.505144 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4750b8-4ce9-4541-a733-d3324b89e198 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:06.505412 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4750b8-4ce9-4541-a733-d3324b89e198 admin admin] PATH_INFO: `/v3/projects/a44c5d1aa5c24288a92072c5181d9166` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:06.506496 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4750b8-4ce9-4541-a733-d3324b89e198 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a44c5d1aa5c24288a92072c5181d9166)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:06.508504 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4750b8-4ce9-4541-a733-d3324b89e198 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:06.556450 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-ee4750b8-4ce9-4541-a733-d3324b89e198 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a44c5d1aa5c24288a92072c5181d9166', 'request_id': 'req-ee4750b8-4ce9-4541-a733-d3324b89e198'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:06.562720 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-ee4750b8-4ce9-4541-a733-d3324b89e198 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a44c5d1aa5c24288a92072c5181d9166', 'request_id': 'req-ee4750b8-4ce9-4541-a733-d3324b89e198'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:06.573033 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-ee4750b8-4ce9-4541-a733-d3324b89e198 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a44c5d1aa5c24288a92072c5181d9166', 'request_id': 'req-ee4750b8-4ce9-4541-a733-d3324b89e198'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:06.585458 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-ee4750b8-4ce9-4541-a733-d3324b89e198 admin admin] The token cache is being invalidate because project a44c5d1aa5c24288a92072c5181d9166 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:51:06.585751 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-ee4750b8-4ce9-4541-a733-d3324b89e198 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:06.586401 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-ee4750b8-4ce9-4541-a733-d3324b89e198 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:06.589291 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1191/2385] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:51:06 2022] DELETE /identity/v3/projects/a44c5d1aa5c24288a92072c5181d9166 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:51:06.592959 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b3716-52b2-424b-9a41-08920c1b269d None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:06.593317 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b3716-52b2-424b-9a41-08920c1b269d None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:06.593749 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b3716-52b2-424b-9a41-08920c1b269d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:06.613699 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-a26b3716-52b2-424b-9a41-08920c1b269d None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:51:06.614879 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-a26b3716-52b2-424b-9a41-08920c1b269d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:06.689008 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1195/2386] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:51:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:51:06.696678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4531cf-8474-4375-9d23-589eff2a3478 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:06.696973 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4531cf-8474-4375-9d23-589eff2a3478 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:06.697254 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4531cf-8474-4375-9d23-589eff2a3478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:06.719694 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-bc4531cf-8474-4375-9d23-589eff2a3478 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:51:06.721281 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-bc4531cf-8474-4375-9d23-589eff2a3478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:06.729521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1192/2387] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:51:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:51:06.734225 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26b3716-52b2-424b-9a41-08920c1b269d None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:06.744008 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26b3716-52b2-424b-9a41-08920c1b269d None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:06.749217 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda909d-f077-4bf6-b3c7-f957d53e3e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:06.749813 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda909d-f077-4bf6-b3c7-f957d53e3e66 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:06.750146 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda909d-f077-4bf6-b3c7-f957d53e3e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:06.750390 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda909d-f077-4bf6-b3c7-f957d53e3e66 admin admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:06.751099 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eda909d-f077-4bf6-b3c7-f957d53e3e66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:51:06.751401 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eda909d-f077-4bf6-b3c7-f957d53e3e66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:06.753097 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eda909d-f077-4bf6-b3c7-f957d53e3e66 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:06.767697 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1196/2388] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:51:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:51:06.772115 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4531cf-8474-4375-9d23-589eff2a3478 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:06.782359 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4531cf-8474-4375-9d23-589eff2a3478 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:06.787655 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd5173-637f-45bf-959e-bcbc772ce0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:06.788286 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd5173-637f-45bf-959e-bcbc772ce0a1 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:06.788565 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd5173-637f-45bf-959e-bcbc772ce0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:06.788834 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd5173-637f-45bf-959e-bcbc772ce0a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:06.789503 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddd5173-637f-45bf-959e-bcbc772ce0a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:06.791451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddd5173-637f-45bf-959e-bcbc772ce0a1 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:06.818565 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1193/2389] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:51:06 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:51:06.822688 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda909d-f077-4bf6-b3c7-f957d53e3e66 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:06.833601 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda909d-f077-4bf6-b3c7-f957d53e3e66 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:06.864576 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:51:06.892514 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77a6932-4e39-417e-9042-234e7302cbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:06.893108 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a6932-4e39-417e-9042-234e7302cbe5 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:06.893383 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a6932-4e39-417e-9042-234e7302cbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:06.893646 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a6932-4e39-417e-9042-234e7302cbe5 admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:06.894264 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77a6932-4e39-417e-9042-234e7302cbe5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:06.896214 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77a6932-4e39-417e-9042-234e7302cbe5 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:06.916553 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1197/2390] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:51:06 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:51:06.921253 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd5173-637f-45bf-959e-bcbc772ce0a1 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:06.931582 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd5173-637f-45bf-959e-bcbc772ce0a1 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:06.936847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8353ef2-4072-49e5-81f7-4f0c71b60258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:06.937443 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8353ef2-4072-49e5-81f7-4f0c71b60258 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:06.937723 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8353ef2-4072-49e5-81f7-4f0c71b60258 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:06.937996 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8353ef2-4072-49e5-81f7-4f0c71b60258 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:06.938639 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8353ef2-4072-49e5-81f7-4f0c71b60258 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:06.940270 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8353ef2-4072-49e5-81f7-4f0c71b60258 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:06.950050 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1194/2391] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:51:06.953818 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77a6932-4e39-417e-9042-234e7302cbe5 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:06.963725 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77a6932-4e39-417e-9042-234e7302cbe5 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:06.968425 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e67df3-4a1e-4806-9660-1ff7665ffd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:06.969013 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e67df3-4a1e-4806-9660-1ff7665ffd15 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:06.969290 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e67df3-4a1e-4806-9660-1ff7665ffd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:06.969565 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e67df3-4a1e-4806-9660-1ff7665ffd15 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:06.970237 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e67df3-4a1e-4806-9660-1ff7665ffd15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:06.971924 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e67df3-4a1e-4806-9660-1ff7665ffd15 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:06.983550 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1198/2392] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:51:06.987727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8353ef2-4072-49e5-81f7-4f0c71b60258 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:06.998860 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8353ef2-4072-49e5-81f7-4f0c71b60258 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:07.003506 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7abdfd-0a02-4ca5-bc34-d28b780ab1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:07.004187 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7abdfd-0a02-4ca5-bc34-d28b780ab1a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:07.004472 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7abdfd-0a02-4ca5-bc34-d28b780ab1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:07.004763 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7abdfd-0a02-4ca5-bc34-d28b780ab1a8 admin admin] PATH_INFO: `/v3/projects/b3cba1a4c0cc476dae7ebece9b4a0d39/users/ab3fa74810c54460ae7ccf2ec244799f/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:07.019066 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7abdfd-0a02-4ca5-bc34-d28b780ab1a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3cba1a4c0cc476dae7ebece9b4a0d39, user_id=ab3fa74810c54460ae7ccf2ec244799f, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:07.021706 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7abdfd-0a02-4ca5-bc34-d28b780ab1a8 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:07.038172 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1195/2393] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:51:06 2022] PUT /identity/v3/projects/b3cba1a4c0cc476dae7ebece9b4a0d39/users/ab3fa74810c54460ae7ccf2ec244799f/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:51:07.050036 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20bab30-21ff-4a4a-a137-9c9e121fbfc4 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:07.050553 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20bab30-21ff-4a4a-a137-9c9e121fbfc4 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:07.050679 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20bab30-21ff-4a4a-a137-9c9e121fbfc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:07.080358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-c20bab30-21ff-4a4a-a137-9c9e121fbfc4 None None] MFA Rules not processed for user `ab3fa74810c54460ae7ccf2ec244799f`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:51:07.081569 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-c20bab30-21ff-4a4a-a137-9c9e121fbfc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:07.152104 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1199/2394] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:51:07 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:51:07.157722 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7abdfd-0a02-4ca5-bc34-d28b780ab1a8 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:07.162536 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:51:07.221848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7abdfd-0a02-4ca5-bc34-d28b780ab1a8 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:07.226566 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa34a6-1f53-4f64-99ca-f46fbd439935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:07.227172 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa34a6-1f53-4f64-99ca-f46fbd439935 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:07.227452 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa34a6-1f53-4f64-99ca-f46fbd439935 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:07.227723 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa34a6-1f53-4f64-99ca-f46fbd439935 admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:07.228370 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa34a6-1f53-4f64-99ca-f46fbd439935 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:07.230290 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa34a6-1f53-4f64-99ca-f46fbd439935 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:07.256693 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1196/2395] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:51:07 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:51:07.260796 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20bab30-21ff-4a4a-a137-9c9e121fbfc4 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:07.272052 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20bab30-21ff-4a4a-a137-9c9e121fbfc4 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:07.329332 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23b4437-13be-46c9-a99d-e70656dbfbbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:07.329948 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b4437-13be-46c9-a99d-e70656dbfbbc admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:07.330212 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b4437-13be-46c9-a99d-e70656dbfbbc admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:07.330521 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b4437-13be-46c9-a99d-e70656dbfbbc admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:07.331140 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23b4437-13be-46c9-a99d-e70656dbfbbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:07.333089 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23b4437-13be-46c9-a99d-e70656dbfbbc admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:07.353539 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1200/2396] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:51:07 2022] POST /identity/v3/users => generated 300 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:51:07.358124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa34a6-1f53-4f64-99ca-f46fbd439935 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:07.368373 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa34a6-1f53-4f64-99ca-f46fbd439935 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:07.373026 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f49657-8112-40de-8b50-090e9f287b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:07.373613 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f49657-8112-40de-8b50-090e9f287b53 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:07.373890 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f49657-8112-40de-8b50-090e9f287b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:07.374161 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f49657-8112-40de-8b50-090e9f287b53 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:07.374793 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f49657-8112-40de-8b50-090e9f287b53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:07.376458 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f49657-8112-40de-8b50-090e9f287b53 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:07.386836 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1197/2397] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:51:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:51:07.390774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23b4437-13be-46c9-a99d-e70656dbfbbc admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:07.400616 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23b4437-13be-46c9-a99d-e70656dbfbbc admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:07.405639 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6894743-654c-4d81-b25b-82edfa8fa145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:07.406272 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6894743-654c-4d81-b25b-82edfa8fa145 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:07.406547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6894743-654c-4d81-b25b-82edfa8fa145 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:07.406818 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6894743-654c-4d81-b25b-82edfa8fa145 admin admin] PATH_INFO: `/v3/projects/f9cb4063fc8240be99f8514a031d043a/users/2f9f53b56b66409fad0ddb0c62780aa5/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:07.422110 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6894743-654c-4d81-b25b-82edfa8fa145 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9cb4063fc8240be99f8514a031d043a, user_id=2f9f53b56b66409fad0ddb0c62780aa5, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:07.424810 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6894743-654c-4d81-b25b-82edfa8fa145 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:07.437770 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1201/2398] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:51:07 2022] PUT /identity/v3/projects/f9cb4063fc8240be99f8514a031d043a/users/2f9f53b56b66409fad0ddb0c62780aa5/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:51:07.441726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f49657-8112-40de-8b50-090e9f287b53 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:07.452444 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f49657-8112-40de-8b50-090e9f287b53 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:07.508369 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0b80ba-69b2-43fe-bca9-69c728891450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:07.508972 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0b80ba-69b2-43fe-bca9-69c728891450 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:07.509251 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0b80ba-69b2-43fe-bca9-69c728891450 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:07.509525 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0b80ba-69b2-43fe-bca9-69c728891450 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:07.510163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0b80ba-69b2-43fe-bca9-69c728891450 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:07.511779 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0b80ba-69b2-43fe-bca9-69c728891450 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:07.521344 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1198/2399] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:51:07 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:51:07.525521 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6894743-654c-4d81-b25b-82edfa8fa145 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:07.536337 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6894743-654c-4d81-b25b-82edfa8fa145 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:07.541242 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87636449-8c84-41fc-a4b7-872175422365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:07.541888 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87636449-8c84-41fc-a4b7-872175422365 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:07.542167 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87636449-8c84-41fc-a4b7-872175422365 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:07.542441 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87636449-8c84-41fc-a4b7-872175422365 admin admin] PATH_INFO: `/v3/projects/f9cb4063fc8240be99f8514a031d043a/users/2f9f53b56b66409fad0ddb0c62780aa5/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:07.545950 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87636449-8c84-41fc-a4b7-872175422365 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9cb4063fc8240be99f8514a031d043a, user_id=2f9f53b56b66409fad0ddb0c62780aa5, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:07.548115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87636449-8c84-41fc-a4b7-872175422365 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:07.550314 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:51:07.560415 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1202/2400] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:51:07 2022] PUT /identity/v3/projects/f9cb4063fc8240be99f8514a031d043a/users/2f9f53b56b66409fad0ddb0c62780aa5/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:51:07.570435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c38bca-3b15-4c1d-a6f9-ee2f014a192e None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:07.570722 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c38bca-3b15-4c1d-a6f9-ee2f014a192e None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:07.570995 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c38bca-3b15-4c1d-a6f9-ee2f014a192e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:07.599118 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-e8c38bca-3b15-4c1d-a6f9-ee2f014a192e None None] MFA Rules not processed for user `2f9f53b56b66409fad0ddb0c62780aa5`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:51:07.600357 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-e8c38bca-3b15-4c1d-a6f9-ee2f014a192e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:07.661614 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:51:07.667831 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1199/2401] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:51:07 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:51:07.678325 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87636449-8c84-41fc-a4b7-872175422365 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:07.680031 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-87636449-8c84-41fc-a4b7-872175422365 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:07.735741 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87636449-8c84-41fc-a4b7-872175422365 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:07.739331 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253dfe2e-085b-4ecc-a3d9-5341280bde9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:07.739916 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253dfe2e-085b-4ecc-a3d9-5341280bde9d service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:07.740218 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253dfe2e-085b-4ecc-a3d9-5341280bde9d service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:07.740486 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253dfe2e-085b-4ecc-a3d9-5341280bde9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:07.749201 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253dfe2e-085b-4ecc-a3d9-5341280bde9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:07.750618 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253dfe2e-085b-4ecc-a3d9-5341280bde9d service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:07.759783 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1203/2402] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:51:07 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:51:07.922902 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c38bca-3b15-4c1d-a6f9-ee2f014a192e None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:07.991824 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c38bca-3b15-4c1d-a6f9-ee2f014a192e None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:07.996938 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c098dc7-3c9f-4f6f-a6da-e1042ac93fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:07.997578 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c098dc7-3c9f-4f6f-a6da-e1042ac93fa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:07.997889 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c098dc7-3c9f-4f6f-a6da-e1042ac93fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:07.998165 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c098dc7-3c9f-4f6f-a6da-e1042ac93fa4 admin admin] PATH_INFO: `/v3/users/ab3fa74810c54460ae7ccf2ec244799f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:07.999301 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c098dc7-3c9f-4f6f-a6da-e1042ac93fa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab3fa74810c54460ae7ccf2ec244799f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:08.001297 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c098dc7-3c9f-4f6f-a6da-e1042ac93fa4 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:08.050205 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-4c098dc7-3c9f-4f6f-a6da-e1042ac93fa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab3fa74810c54460ae7ccf2ec244799f'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:08.050856 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-4c098dc7-3c9f-4f6f-a6da-e1042ac93fa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab3fa74810c54460ae7ccf2ec244799f'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:08.051450 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-4c098dc7-3c9f-4f6f-a6da-e1042ac93fa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab3fa74810c54460ae7ccf2ec244799f'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:08.060474 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-4c098dc7-3c9f-4f6f-a6da-e1042ac93fa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab3fa74810c54460ae7ccf2ec244799f'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:08.067730 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-4c098dc7-3c9f-4f6f-a6da-e1042ac93fa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab3fa74810c54460ae7ccf2ec244799f'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:08.086891 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1200/2403] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:51:07 2022] DELETE /identity/v3/users/ab3fa74810c54460ae7ccf2ec244799f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:51:08.091677 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7abbd-7ebd-4d0f-ac4e-4080c38bef68 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:08.092174 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7abbd-7ebd-4d0f-ac4e-4080c38bef68 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:08.092286 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7abbd-7ebd-4d0f-ac4e-4080c38bef68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:08.113511 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-0ee7abbd-7ebd-4d0f-ac4e-4080c38bef68 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:51:08.114717 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-0ee7abbd-7ebd-4d0f-ac4e-4080c38bef68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:08.176968 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1204/2404] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:51:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:51:08.187132 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c098dc7-3c9f-4f6f-a6da-e1042ac93fa4 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:08.188661 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-4c098dc7-3c9f-4f6f-a6da-e1042ac93fa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:08.245635 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c098dc7-3c9f-4f6f-a6da-e1042ac93fa4 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:08.249832 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff0f02-b8a4-408b-9323-22fbe4540a57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:08.250435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff0f02-b8a4-408b-9323-22fbe4540a57 service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:08.250718 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff0f02-b8a4-408b-9323-22fbe4540a57 service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:08.251346 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff0f02-b8a4-408b-9323-22fbe4540a57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:08.260191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff0f02-b8a4-408b-9323-22fbe4540a57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:08.261619 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff0f02-b8a4-408b-9323-22fbe4540a57 service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:08.272080 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1201/2405] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:51:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:51:08.539071 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7abbd-7ebd-4d0f-ac4e-4080c38bef68 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:08.540535 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-0ee7abbd-7ebd-4d0f-ac4e-4080c38bef68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:08.552858 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7abbd-7ebd-4d0f-ac4e-4080c38bef68 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:08.557504 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96558d17-d7e9-46cc-84f5-6dcac0105adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:08.558044 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96558d17-d7e9-46cc-84f5-6dcac0105adc admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:08.558320 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96558d17-d7e9-46cc-84f5-6dcac0105adc admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:08.558670 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96558d17-d7e9-46cc-84f5-6dcac0105adc admin admin] PATH_INFO: `/v3/projects/b3cba1a4c0cc476dae7ebece9b4a0d39` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:08.560519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96558d17-d7e9-46cc-84f5-6dcac0105adc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3cba1a4c0cc476dae7ebece9b4a0d39)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:08.562453 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96558d17-d7e9-46cc-84f5-6dcac0105adc admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:08.609126 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-96558d17-d7e9-46cc-84f5-6dcac0105adc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3cba1a4c0cc476dae7ebece9b4a0d39', 'request_id': 'req-96558d17-d7e9-46cc-84f5-6dcac0105adc'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:08.614625 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-96558d17-d7e9-46cc-84f5-6dcac0105adc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3cba1a4c0cc476dae7ebece9b4a0d39', 'request_id': 'req-96558d17-d7e9-46cc-84f5-6dcac0105adc'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:08.622510 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-96558d17-d7e9-46cc-84f5-6dcac0105adc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3cba1a4c0cc476dae7ebece9b4a0d39', 'request_id': 'req-96558d17-d7e9-46cc-84f5-6dcac0105adc'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:08.633490 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-96558d17-d7e9-46cc-84f5-6dcac0105adc admin admin] The token cache is being invalidate because project b3cba1a4c0cc476dae7ebece9b4a0d39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:51:08.633773 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-96558d17-d7e9-46cc-84f5-6dcac0105adc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:08.634392 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-96558d17-d7e9-46cc-84f5-6dcac0105adc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:08.635873 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1205/2406] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:51:08 2022] DELETE /identity/v3/projects/b3cba1a4c0cc476dae7ebece9b4a0d39 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:51:08.639780 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff0f02-b8a4-408b-9323-22fbe4540a57 service neutron] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:08.649520 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff0f02-b8a4-408b-9323-22fbe4540a57 service neutron] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:08.666073 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:51:08.708503 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ac473-e6e6-47cc-975a-f4119504e637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:08.709128 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ac473-e6e6-47cc-975a-f4119504e637 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:08.709402 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ac473-e6e6-47cc-975a-f4119504e637 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:08.709677 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ac473-e6e6-47cc-975a-f4119504e637 admin admin] PATH_INFO: `/v3/users/2f9f53b56b66409fad0ddb0c62780aa5` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:08.710777 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1ac473-e6e6-47cc-975a-f4119504e637 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f9f53b56b66409fad0ddb0c62780aa5)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:08.712717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1ac473-e6e6-47cc-975a-f4119504e637 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:08.757527 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-9d1ac473-e6e6-47cc-975a-f4119504e637 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f9f53b56b66409fad0ddb0c62780aa5'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:08.758172 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-9d1ac473-e6e6-47cc-975a-f4119504e637 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f9f53b56b66409fad0ddb0c62780aa5'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:08.758779 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-9d1ac473-e6e6-47cc-975a-f4119504e637 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f9f53b56b66409fad0ddb0c62780aa5'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:08.765949 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-9d1ac473-e6e6-47cc-975a-f4119504e637 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f9f53b56b66409fad0ddb0c62780aa5'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:08.772997 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-9d1ac473-e6e6-47cc-975a-f4119504e637 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f9f53b56b66409fad0ddb0c62780aa5'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:08.790733 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1202/2407] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:51:08 2022] DELETE /identity/v3/users/2f9f53b56b66409fad0ddb0c62780aa5 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:51:09.060378 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96558d17-d7e9-46cc-84f5-6dcac0105adc admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:09.061732 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-96558d17-d7e9-46cc-84f5-6dcac0105adc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:09.077980 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96558d17-d7e9-46cc-84f5-6dcac0105adc admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:09.096781 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:51:09.139928 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47136703-153e-4a66-aac8-0e2f41c934f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:09.140493 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47136703-153e-4a66-aac8-0e2f41c934f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:09.140869 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47136703-153e-4a66-aac8-0e2f41c934f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:09.141086 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47136703-153e-4a66-aac8-0e2f41c934f3 admin admin] PATH_INFO: `/v3/projects/f9cb4063fc8240be99f8514a031d043a` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:09.142209 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47136703-153e-4a66-aac8-0e2f41c934f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9cb4063fc8240be99f8514a031d043a)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:09.144150 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47136703-153e-4a66-aac8-0e2f41c934f3 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:09.191515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-47136703-153e-4a66-aac8-0e2f41c934f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9cb4063fc8240be99f8514a031d043a', 'request_id': 'req-47136703-153e-4a66-aac8-0e2f41c934f3'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:09.197587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-47136703-153e-4a66-aac8-0e2f41c934f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9cb4063fc8240be99f8514a031d043a', 'request_id': 'req-47136703-153e-4a66-aac8-0e2f41c934f3'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:09.205441 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-47136703-153e-4a66-aac8-0e2f41c934f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9cb4063fc8240be99f8514a031d043a', 'request_id': 'req-47136703-153e-4a66-aac8-0e2f41c934f3'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:09.217430 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-47136703-153e-4a66-aac8-0e2f41c934f3 admin admin] The token cache is being invalidate because project f9cb4063fc8240be99f8514a031d043a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:51:09.217724 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-47136703-153e-4a66-aac8-0e2f41c934f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:09.218380 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-47136703-153e-4a66-aac8-0e2f41c934f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:09.220135 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1206/2408] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:51:09 2022] DELETE /identity/v3/projects/f9cb4063fc8240be99f8514a031d043a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:51:09.224874 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321bb43-cb06-47c0-a91c-d4f5e8b1988e None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:09.225310 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321bb43-cb06-47c0-a91c-d4f5e8b1988e None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:09.225441 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321bb43-cb06-47c0-a91c-d4f5e8b1988e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:09.244822 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-b321bb43-cb06-47c0-a91c-d4f5e8b1988e None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:51:09.246143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-b321bb43-cb06-47c0-a91c-d4f5e8b1988e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:09.311107 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1203/2409] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:51:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:51:09.319080 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d30e1-bbf5-46b6-9da1-eb78cb23f9cf None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:09.319364 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d30e1-bbf5-46b6-9da1-eb78cb23f9cf None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:09.319637 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d30e1-bbf5-46b6-9da1-eb78cb23f9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:09.339456 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-347d30e1-bbf5-46b6-9da1-eb78cb23f9cf None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:51:09.340698 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-347d30e1-bbf5-46b6-9da1-eb78cb23f9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:09.349486 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1207/2410] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:51:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:51:09.353971 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b321bb43-cb06-47c0-a91c-d4f5e8b1988e None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:09.363786 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b321bb43-cb06-47c0-a91c-d4f5e8b1988e None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:09.368608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d1b4c9-2984-403c-9367-956b5d7872c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:09.369215 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1b4c9-2984-403c-9367-956b5d7872c4 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:09.369495 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1b4c9-2984-403c-9367-956b5d7872c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:09.369769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1b4c9-2984-403c-9367-956b5d7872c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:09.370422 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d1b4c9-2984-403c-9367-956b5d7872c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:51:09.370726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d1b4c9-2984-403c-9367-956b5d7872c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:09.372389 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d1b4c9-2984-403c-9367-956b5d7872c4 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:09.384293 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1204/2411] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:51:09.389065 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347d30e1-bbf5-46b6-9da1-eb78cb23f9cf None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:09.398147 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347d30e1-bbf5-46b6-9da1-eb78cb23f9cf None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:09.402688 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1fcba3-f2f2-4566-bd3f-83c3569b567a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:09.403281 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1fcba3-f2f2-4566-bd3f-83c3569b567a admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:09.403561 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1fcba3-f2f2-4566-bd3f-83c3569b567a admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:09.403830 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1fcba3-f2f2-4566-bd3f-83c3569b567a admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:09.405032 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1fcba3-f2f2-4566-bd3f-83c3569b567a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:09.406935 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1fcba3-f2f2-4566-bd3f-83c3569b567a admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:09.437223 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1208/2412] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:51:09 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:51:09.438298 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d1b4c9-2984-403c-9367-956b5d7872c4 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:09.449042 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d1b4c9-2984-403c-9367-956b5d7872c4 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:09.475788 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:51:09.513222 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea35d08-16c5-4465-9653-990c8315a34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:09.514413 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea35d08-16c5-4465-9653-990c8315a34e admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:09.514693 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea35d08-16c5-4465-9653-990c8315a34e admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:09.514964 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea35d08-16c5-4465-9653-990c8315a34e admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:09.515591 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea35d08-16c5-4465-9653-990c8315a34e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:09.517570 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea35d08-16c5-4465-9653-990c8315a34e admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:09.538115 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1205/2413] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:51:09 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:51:09.542329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1fcba3-f2f2-4566-bd3f-83c3569b567a admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:09.552465 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1fcba3-f2f2-4566-bd3f-83c3569b567a admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:09.556954 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf71358-04da-4fab-9d5b-4170ae3af436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:09.557544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf71358-04da-4fab-9d5b-4170ae3af436 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:09.557821 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf71358-04da-4fab-9d5b-4170ae3af436 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:09.558089 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf71358-04da-4fab-9d5b-4170ae3af436 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:09.558729 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf71358-04da-4fab-9d5b-4170ae3af436 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:09.560917 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf71358-04da-4fab-9d5b-4170ae3af436 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:09.571950 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1209/2414] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:51:09.575688 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea35d08-16c5-4465-9653-990c8315a34e admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:09.586170 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea35d08-16c5-4465-9653-990c8315a34e admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:09.590798 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3830102-70c9-4fcc-b228-38e81314f364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:09.591413 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3830102-70c9-4fcc-b228-38e81314f364 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:09.591692 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3830102-70c9-4fcc-b228-38e81314f364 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:09.591990 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3830102-70c9-4fcc-b228-38e81314f364 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:09.592630 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3830102-70c9-4fcc-b228-38e81314f364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:09.594210 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3830102-70c9-4fcc-b228-38e81314f364 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:09.604812 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1206/2415] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 23:51:09.608879 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf71358-04da-4fab-9d5b-4170ae3af436 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:09.619507 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf71358-04da-4fab-9d5b-4170ae3af436 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:09.624902 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da15ab1-3237-4e6f-a253-c477415dad81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:09.625545 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da15ab1-3237-4e6f-a253-c477415dad81 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:09.625821 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da15ab1-3237-4e6f-a253-c477415dad81 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:09.626119 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da15ab1-3237-4e6f-a253-c477415dad81 admin admin] PATH_INFO: `/v3/projects/3eeb37f74c1a4a06ae48dbde8dc7f1f2/users/b0b18f69203c4f328df4bf61e0277d7c/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:09.641307 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da15ab1-3237-4e6f-a253-c477415dad81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eeb37f74c1a4a06ae48dbde8dc7f1f2, user_id=b0b18f69203c4f328df4bf61e0277d7c, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:09.644028 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da15ab1-3237-4e6f-a253-c477415dad81 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:09.657507 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1210/2416] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:51:09 2022] PUT /identity/v3/projects/3eeb37f74c1a4a06ae48dbde8dc7f1f2/users/b0b18f69203c4f328df4bf61e0277d7c/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:51:09.669763 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd37f65e-74be-40f7-946d-800eb52ae2ed None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:09.670105 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd37f65e-74be-40f7-946d-800eb52ae2ed None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:09.670389 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd37f65e-74be-40f7-946d-800eb52ae2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:09.698861 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-cd37f65e-74be-40f7-946d-800eb52ae2ed None None] MFA Rules not processed for user `b0b18f69203c4f328df4bf61e0277d7c`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:51:09.700722 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-cd37f65e-74be-40f7-946d-800eb52ae2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:09.769727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1207/2417] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:51:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:51:09.781074 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da15ab1-3237-4e6f-a253-c477415dad81 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:09.782828 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-6da15ab1-3237-4e6f-a253-c477415dad81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:09.836712 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:51:09.843579 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da15ab1-3237-4e6f-a253-c477415dad81 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:09.848315 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3ca8f-cb1c-496d-84f1-bb6023e4423e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:09.848954 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3ca8f-cb1c-496d-84f1-bb6023e4423e service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:09.849231 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3ca8f-cb1c-496d-84f1-bb6023e4423e service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:09.849498 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3ca8f-cb1c-496d-84f1-bb6023e4423e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:09.858257 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b3ca8f-cb1c-496d-84f1-bb6023e4423e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:09.859682 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b3ca8f-cb1c-496d-84f1-bb6023e4423e service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:09.868232 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1211/2418] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:51:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:51:09.890077 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd37f65e-74be-40f7-946d-800eb52ae2ed None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:09.950377 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd37f65e-74be-40f7-946d-800eb52ae2ed None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:09.954892 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2166e2-0b1a-4ec6-a0da-a8bae3906bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:09.955508 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2166e2-0b1a-4ec6-a0da-a8bae3906bf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:09.955784 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2166e2-0b1a-4ec6-a0da-a8bae3906bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:09.956163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2166e2-0b1a-4ec6-a0da-a8bae3906bf3 admin admin] PATH_INFO: `/v3/users/b0b18f69203c4f328df4bf61e0277d7c` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:09.957276 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2166e2-0b1a-4ec6-a0da-a8bae3906bf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0b18f69203c4f328df4bf61e0277d7c)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:09.959190 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2166e2-0b1a-4ec6-a0da-a8bae3906bf3 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:09.995585 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:51:10.006061 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-7a2166e2-0b1a-4ec6-a0da-a8bae3906bf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0b18f69203c4f328df4bf61e0277d7c'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:10.006706 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-7a2166e2-0b1a-4ec6-a0da-a8bae3906bf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0b18f69203c4f328df4bf61e0277d7c'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:10.007349 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-7a2166e2-0b1a-4ec6-a0da-a8bae3906bf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0b18f69203c4f328df4bf61e0277d7c'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:10.014489 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-7a2166e2-0b1a-4ec6-a0da-a8bae3906bf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0b18f69203c4f328df4bf61e0277d7c'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:10.021363 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-7a2166e2-0b1a-4ec6-a0da-a8bae3906bf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0b18f69203c4f328df4bf61e0277d7c'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:10.038749 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1208/2419] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:51:09 2022] DELETE /identity/v3/users/b0b18f69203c4f328df4bf61e0277d7c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:51:10.045862 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6522f-656d-4901-8537-f7c8b1f6e4f5 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:10.046219 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6522f-656d-4901-8537-f7c8b1f6e4f5 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:10.046500 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6522f-656d-4901-8537-f7c8b1f6e4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:10.067478 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-bbb6522f-656d-4901-8537-f7c8b1f6e4f5 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:51:10.068709 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-bbb6522f-656d-4901-8537-f7c8b1f6e4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:10.126921 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1212/2420] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:51:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:51:10.136513 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2166e2-0b1a-4ec6-a0da-a8bae3906bf3 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:10.137912 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-7a2166e2-0b1a-4ec6-a0da-a8bae3906bf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:10.197631 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2166e2-0b1a-4ec6-a0da-a8bae3906bf3 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:10.201317 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87bef39-f23b-49cf-bbed-7f911e6b14a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:10.201917 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87bef39-f23b-49cf-bbed-7f911e6b14a4 service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:10.202198 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87bef39-f23b-49cf-bbed-7f911e6b14a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:10.202475 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87bef39-f23b-49cf-bbed-7f911e6b14a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:10.211355 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87bef39-f23b-49cf-bbed-7f911e6b14a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:10.212823 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87bef39-f23b-49cf-bbed-7f911e6b14a4 service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:10.223061 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1209/2421] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:51:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:51:10.549672 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb6522f-656d-4901-8537-f7c8b1f6e4f5 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:10.551260 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-bbb6522f-656d-4901-8537-f7c8b1f6e4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:10.564086 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb6522f-656d-4901-8537-f7c8b1f6e4f5 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:10.570440 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ebb8fa-7039-406d-a4d1-6d900b032dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:10.571099 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ebb8fa-7039-406d-a4d1-6d900b032dd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:10.571376 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ebb8fa-7039-406d-a4d1-6d900b032dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:10.571650 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ebb8fa-7039-406d-a4d1-6d900b032dd5 admin admin] PATH_INFO: `/v3/projects/3eeb37f74c1a4a06ae48dbde8dc7f1f2` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:10.572856 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ebb8fa-7039-406d-a4d1-6d900b032dd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3eeb37f74c1a4a06ae48dbde8dc7f1f2)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:10.574976 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ebb8fa-7039-406d-a4d1-6d900b032dd5 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:10.623320 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-71ebb8fa-7039-406d-a4d1-6d900b032dd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3eeb37f74c1a4a06ae48dbde8dc7f1f2', 'request_id': 'req-71ebb8fa-7039-406d-a4d1-6d900b032dd5'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:10.628727 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-71ebb8fa-7039-406d-a4d1-6d900b032dd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3eeb37f74c1a4a06ae48dbde8dc7f1f2', 'request_id': 'req-71ebb8fa-7039-406d-a4d1-6d900b032dd5'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:10.636440 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-71ebb8fa-7039-406d-a4d1-6d900b032dd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3eeb37f74c1a4a06ae48dbde8dc7f1f2', 'request_id': 'req-71ebb8fa-7039-406d-a4d1-6d900b032dd5'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:10.648920 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-71ebb8fa-7039-406d-a4d1-6d900b032dd5 admin admin] The token cache is being invalidate because project 3eeb37f74c1a4a06ae48dbde8dc7f1f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:51:10.649139 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-71ebb8fa-7039-406d-a4d1-6d900b032dd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:10.649773 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-71ebb8fa-7039-406d-a4d1-6d900b032dd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:10.651468 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1213/2422] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:51:10 2022] DELETE /identity/v3/projects/3eeb37f74c1a4a06ae48dbde8dc7f1f2 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:51:10.656772 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a6309-e3b5-4751-94dd-4da90f19b5a5 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:10.657257 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a6309-e3b5-4751-94dd-4da90f19b5a5 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:10.657475 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a6309-e3b5-4751-94dd-4da90f19b5a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:10.680856 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-a25a6309-e3b5-4751-94dd-4da90f19b5a5 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:51:10.680856 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-a25a6309-e3b5-4751-94dd-4da90f19b5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:10.742287 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1210/2423] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:51:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:51:10.749996 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f218bb-93c3-41eb-9312-4857088ac657 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:10.750280 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f218bb-93c3-41eb-9312-4857088ac657 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:10.750550 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f218bb-93c3-41eb-9312-4857088ac657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:10.773682 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-d4f218bb-93c3-41eb-9312-4857088ac657 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:51:10.774884 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-d4f218bb-93c3-41eb-9312-4857088ac657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:10.786908 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1214/2424] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:51:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:51:10.794152 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a6309-e3b5-4751-94dd-4da90f19b5a5 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:10.806478 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a6309-e3b5-4751-94dd-4da90f19b5a5 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:10.811604 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df8fb84-bbc8-49d2-af8c-a6819206e6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:10.812247 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8fb84-bbc8-49d2-af8c-a6819206e6f2 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:10.812537 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8fb84-bbc8-49d2-af8c-a6819206e6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:10.812797 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8fb84-bbc8-49d2-af8c-a6819206e6f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:10.813458 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df8fb84-bbc8-49d2-af8c-a6819206e6f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:51:10.813743 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df8fb84-bbc8-49d2-af8c-a6819206e6f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:10.815330 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df8fb84-bbc8-49d2-af8c-a6819206e6f2 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:10.829150 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1211/2425] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:51:10.833883 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f218bb-93c3-41eb-9312-4857088ac657 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:10.849187 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f218bb-93c3-41eb-9312-4857088ac657 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:10.860509 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658cf03-6fd5-45ac-aaa9-8e4255a9ecc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:10.861242 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658cf03-6fd5-45ac-aaa9-8e4255a9ecc7 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:10.861589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658cf03-6fd5-45ac-aaa9-8e4255a9ecc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:10.861942 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658cf03-6fd5-45ac-aaa9-8e4255a9ecc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:10.862628 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658cf03-6fd5-45ac-aaa9-8e4255a9ecc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:10.864676 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658cf03-6fd5-45ac-aaa9-8e4255a9ecc7 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:10.895703 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1215/2426] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:51:10 2022] POST /identity/v3/projects => generated 371 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:51:10.899750 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df8fb84-bbc8-49d2-af8c-a6819206e6f2 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:10.910165 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df8fb84-bbc8-49d2-af8c-a6819206e6f2 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:10.952614 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:51:10.971907 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493731a8-96cd-4010-8260-6bde1d2d824c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:10.972557 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493731a8-96cd-4010-8260-6bde1d2d824c admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:10.972837 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493731a8-96cd-4010-8260-6bde1d2d824c admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:10.973108 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493731a8-96cd-4010-8260-6bde1d2d824c admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:10.973754 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493731a8-96cd-4010-8260-6bde1d2d824c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:10.975689 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493731a8-96cd-4010-8260-6bde1d2d824c admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:10.999942 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1212/2427] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:51:10 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:51:11.004225 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658cf03-6fd5-45ac-aaa9-8e4255a9ecc7 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:11.021674 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658cf03-6fd5-45ac-aaa9-8e4255a9ecc7 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:11.026609 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c89546-118f-4398-a804-6ba31a65007d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:11.027213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c89546-118f-4398-a804-6ba31a65007d admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:11.027496 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c89546-118f-4398-a804-6ba31a65007d admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:11.027819 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c89546-118f-4398-a804-6ba31a65007d admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:11.028505 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c89546-118f-4398-a804-6ba31a65007d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:11.030591 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c89546-118f-4398-a804-6ba31a65007d admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:11.042296 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1216/2428] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 23:51:11.045683 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493731a8-96cd-4010-8260-6bde1d2d824c admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:11.056205 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493731a8-96cd-4010-8260-6bde1d2d824c admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:11.061733 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea9e0d2-b2ce-4962-b5fc-effd544e3907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:11.062445 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9e0d2-b2ce-4962-b5fc-effd544e3907 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:11.062823 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9e0d2-b2ce-4962-b5fc-effd544e3907 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:11.063220 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9e0d2-b2ce-4962-b5fc-effd544e3907 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:11.064171 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea9e0d2-b2ce-4962-b5fc-effd544e3907 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:11.065953 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea9e0d2-b2ce-4962-b5fc-effd544e3907 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:11.078288 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1213/2429] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:51:11.082463 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c89546-118f-4398-a804-6ba31a65007d admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:11.094254 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c89546-118f-4398-a804-6ba31a65007d admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:11.099234 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127f3e8-0c62-4995-84a9-ac7ac5ad2e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:11.100081 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127f3e8-0c62-4995-84a9-ac7ac5ad2e0e admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:11.100460 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127f3e8-0c62-4995-84a9-ac7ac5ad2e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:11.100829 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127f3e8-0c62-4995-84a9-ac7ac5ad2e0e admin admin] PATH_INFO: `/v3/projects/2ccde22a7c8a4a9bbfaab1dad9240f34/users/185b05ace78d4e369cb0354a0865fd52/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:11.118242 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127f3e8-0c62-4995-84a9-ac7ac5ad2e0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ccde22a7c8a4a9bbfaab1dad9240f34, user_id=185b05ace78d4e369cb0354a0865fd52, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:11.121182 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127f3e8-0c62-4995-84a9-ac7ac5ad2e0e admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:11.140467 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1217/2430] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:51:11 2022] PUT /identity/v3/projects/2ccde22a7c8a4a9bbfaab1dad9240f34/users/185b05ace78d4e369cb0354a0865fd52/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:51:11.145537 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90adfbe-fc56-426c-90aa-0523d64ac642 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:11.145839 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90adfbe-fc56-426c-90aa-0523d64ac642 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:11.146110 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90adfbe-fc56-426c-90aa-0523d64ac642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:11.166484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-a90adfbe-fc56-426c-90aa-0523d64ac642 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:51:11.167721 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-a90adfbe-fc56-426c-90aa-0523d64ac642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:11.238257 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1214/2431] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:51:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:51:11.248547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127f3e8-0c62-4995-84a9-ac7ac5ad2e0e admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:11.250296 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-5127f3e8-0c62-4995-84a9-ac7ac5ad2e0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:11.262414 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:51:11.323920 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127f3e8-0c62-4995-84a9-ac7ac5ad2e0e admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:11.328881 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930fc1c3-3143-4c59-a741-d1fed7724a88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:11.329715 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fc1c3-3143-4c59-a741-d1fed7724a88 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:11.329997 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fc1c3-3143-4c59-a741-d1fed7724a88 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:11.330430 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fc1c3-3143-4c59-a741-d1fed7724a88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:11.347130 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930fc1c3-3143-4c59-a741-d1fed7724a88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:11.363710 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930fc1c3-3143-4c59-a741-d1fed7724a88 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:11.381276 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1218/2432] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:51:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:51:12.057861 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139484a1-0ebe-42cd-a627-ee878a440ea1 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:12.058401 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139484a1-0ebe-42cd-a627-ee878a440ea1 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:12.058594 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139484a1-0ebe-42cd-a627-ee878a440ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:12.089723 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-139484a1-0ebe-42cd-a627-ee878a440ea1 None None] MFA Rules not processed for user `185b05ace78d4e369cb0354a0865fd52`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:51:12.090960 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-139484a1-0ebe-42cd-a627-ee878a440ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:12.168745 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1215/2433] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:51:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:51:12.180089 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930fc1c3-3143-4c59-a741-d1fed7724a88 service neutron] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:12.182042 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-930fc1c3-3143-4c59-a741-d1fed7724a88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:12.250085 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930fc1c3-3143-4c59-a741-d1fed7724a88 service neutron] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:12.255267 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf837a5f-578a-4600-b41b-c1aad55595da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:12.255879 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf837a5f-578a-4600-b41b-c1aad55595da service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:12.256191 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf837a5f-578a-4600-b41b-c1aad55595da service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:12.256462 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf837a5f-578a-4600-b41b-c1aad55595da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:12.266354 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf837a5f-578a-4600-b41b-c1aad55595da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:12.267786 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf837a5f-578a-4600-b41b-c1aad55595da service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:12.277429 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1219/2434] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:51:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:51:12.738803 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01b8e0-554c-4d8a-99c0-961dd52da715 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:12.739097 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01b8e0-554c-4d8a-99c0-961dd52da715 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:12.739378 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01b8e0-554c-4d8a-99c0-961dd52da715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:12.760651 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-7c01b8e0-554c-4d8a-99c0-961dd52da715 None None] MFA Rules not processed for user `d0a13dbffb4a49508b7fb661cf84e803`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:51:12.761863 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-7c01b8e0-554c-4d8a-99c0-961dd52da715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:12.809192 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:51:12.819338 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1216/2435] 10.222.0.26 () {58 vars in 967 bytes} [Wed Aug 17 23:51:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:51:12.864715 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf837a5f-578a-4600-b41b-c1aad55595da service cinder] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:12.866139 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-cf837a5f-578a-4600-b41b-c1aad55595da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:12.936071 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf837a5f-578a-4600-b41b-c1aad55595da service cinder] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:12.941029 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a7958b-189b-44e3-8adc-a4a4ee808972 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee422ea90557444fabdc9c292865b9b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:12.941637 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7958b-189b-44e3-8adc-a4a4ee808972 service swift] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:12.941916 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7958b-189b-44e3-8adc-a4a4ee808972 service swift] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:12.942189 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7958b-189b-44e3-8adc-a4a4ee808972 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:12.950068 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a7958b-189b-44e3-8adc-a4a4ee808972 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:12.951654 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a7958b-189b-44e3-8adc-a4a4ee808972 service swift] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:12.959576 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1220/2436] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Aug 17 23:51:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 23:51:13.718933 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ccb748-df08-4276-bd1c-a41931d1971e None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:13.719490 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ccb748-df08-4276-bd1c-a41931d1971e None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:13.719490 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ccb748-df08-4276-bd1c-a41931d1971e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:13.738408 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-f7ccb748-df08-4276-bd1c-a41931d1971e None None] MFA Rules not processed for user `185b05ace78d4e369cb0354a0865fd52`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:51:13.739770 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-f7ccb748-df08-4276-bd1c-a41931d1971e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:13.746991 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1217/2437] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:51:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:51:20.307534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a7958b-189b-44e3-8adc-a4a4ee808972 service swift] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:20.309018 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-d4a7958b-189b-44e3-8adc-a4a4ee808972 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:20.359804 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:51:20.388234 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a7958b-189b-44e3-8adc-a4a4ee808972 service swift] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:20.392812 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b274b-a0ca-4932-8654-9d8f514a94ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:20.393432 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b274b-a0ca-4932-8654-9d8f514a94ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:20.393709 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b274b-a0ca-4932-8654-9d8f514a94ed admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:20.393977 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b274b-a0ca-4932-8654-9d8f514a94ed admin admin] PATH_INFO: `/v3/users/006f31b802f94549a517df90180dca36` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:20.395100 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497b274b-a0ca-4932-8654-9d8f514a94ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=006f31b802f94549a517df90180dca36)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:20.397071 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497b274b-a0ca-4932-8654-9d8f514a94ed admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:20.442235 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-497b274b-a0ca-4932-8654-9d8f514a94ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '006f31b802f94549a517df90180dca36'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:20.442900 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-497b274b-a0ca-4932-8654-9d8f514a94ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '006f31b802f94549a517df90180dca36'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:20.455193 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-497b274b-a0ca-4932-8654-9d8f514a94ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '006f31b802f94549a517df90180dca36'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:20.462543 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-497b274b-a0ca-4932-8654-9d8f514a94ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '006f31b802f94549a517df90180dca36'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:20.463165 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-497b274b-a0ca-4932-8654-9d8f514a94ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '006f31b802f94549a517df90180dca36'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:20.475144 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1221/2438] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:51:20 2022] DELETE /identity/v3/users/006f31b802f94549a517df90180dca36 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:51:20.754358 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ccb748-df08-4276-bd1c-a41931d1971e None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:20.798696 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ccb748-df08-4276-bd1c-a41931d1971e None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:20.855249 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86162275-7ef0-4fa5-a99e-c9a6b6448ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:20.855874 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86162275-7ef0-4fa5-a99e-c9a6b6448ab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:20.856178 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86162275-7ef0-4fa5-a99e-c9a6b6448ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:20.856448 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86162275-7ef0-4fa5-a99e-c9a6b6448ab9 admin admin] PATH_INFO: `/v3/projects/b3ff5d4d64a740cabdc0144fe0b475ea` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:20.858228 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86162275-7ef0-4fa5-a99e-c9a6b6448ab9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3ff5d4d64a740cabdc0144fe0b475ea)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:20.860163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86162275-7ef0-4fa5-a99e-c9a6b6448ab9 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:20.907657 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-86162275-7ef0-4fa5-a99e-c9a6b6448ab9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3ff5d4d64a740cabdc0144fe0b475ea', 'request_id': 'req-86162275-7ef0-4fa5-a99e-c9a6b6448ab9'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:20.913880 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-86162275-7ef0-4fa5-a99e-c9a6b6448ab9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3ff5d4d64a740cabdc0144fe0b475ea', 'request_id': 'req-86162275-7ef0-4fa5-a99e-c9a6b6448ab9'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:20.926729 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-86162275-7ef0-4fa5-a99e-c9a6b6448ab9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3ff5d4d64a740cabdc0144fe0b475ea', 'request_id': 'req-86162275-7ef0-4fa5-a99e-c9a6b6448ab9'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:20.938372 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-86162275-7ef0-4fa5-a99e-c9a6b6448ab9 admin admin] The token cache is being invalidate because project b3ff5d4d64a740cabdc0144fe0b475ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:51:20.938673 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-86162275-7ef0-4fa5-a99e-c9a6b6448ab9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:20.939298 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-86162275-7ef0-4fa5-a99e-c9a6b6448ab9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:20.940807 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1218/2439] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:51:20 2022] DELETE /identity/v3/projects/b3ff5d4d64a740cabdc0144fe0b475ea => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:51:20.944882 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b274b-a0ca-4932-8654-9d8f514a94ed admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:20.946284 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-497b274b-a0ca-4932-8654-9d8f514a94ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:20.959926 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b274b-a0ca-4932-8654-9d8f514a94ed admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:21.020233 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccc17b1-27c8-4255-864a-d2d2b95e3874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:21.020858 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccc17b1-27c8-4255-864a-d2d2b95e3874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:21.021133 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccc17b1-27c8-4255-864a-d2d2b95e3874 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:21.021399 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccc17b1-27c8-4255-864a-d2d2b95e3874 admin admin] PATH_INFO: `/v3/users/41b0eeaee3724e7f8d3c4b63c298d5e6` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:21.022497 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccc17b1-27c8-4255-864a-d2d2b95e3874 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41b0eeaee3724e7f8d3c4b63c298d5e6)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:21.024552 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccc17b1-27c8-4255-864a-d2d2b95e3874 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:21.070260 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-cccc17b1-27c8-4255-864a-d2d2b95e3874 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41b0eeaee3724e7f8d3c4b63c298d5e6'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:21.070902 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-cccc17b1-27c8-4255-864a-d2d2b95e3874 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41b0eeaee3724e7f8d3c4b63c298d5e6'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:21.078362 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:51:21.085780 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-cccc17b1-27c8-4255-864a-d2d2b95e3874 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41b0eeaee3724e7f8d3c4b63c298d5e6'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:21.095721 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-cccc17b1-27c8-4255-864a-d2d2b95e3874 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41b0eeaee3724e7f8d3c4b63c298d5e6'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:21.096425 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-cccc17b1-27c8-4255-864a-d2d2b95e3874 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41b0eeaee3724e7f8d3c4b63c298d5e6'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:21.110085 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1222/2440] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:51:20 2022] DELETE /identity/v3/users/41b0eeaee3724e7f8d3c4b63c298d5e6 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:51:21.289750 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86162275-7ef0-4fa5-a99e-c9a6b6448ab9 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:21.291175 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-86162275-7ef0-4fa5-a99e-c9a6b6448ab9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:21.303635 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86162275-7ef0-4fa5-a99e-c9a6b6448ab9 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:21.365937 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb710a0-1de6-4a4e-a8f6-2719cf0dc00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:21.366573 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb710a0-1de6-4a4e-a8f6-2719cf0dc00e admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:21.366851 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb710a0-1de6-4a4e-a8f6-2719cf0dc00e admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:21.367123 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb710a0-1de6-4a4e-a8f6-2719cf0dc00e admin admin] PATH_INFO: `/v3/projects/3c5b125928c1443f86967c0f7c56fcec` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:21.368293 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb710a0-1de6-4a4e-a8f6-2719cf0dc00e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c5b125928c1443f86967c0f7c56fcec)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:21.370228 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb710a0-1de6-4a4e-a8f6-2719cf0dc00e admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:21.376583 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:51:21.417520 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-2fb710a0-1de6-4a4e-a8f6-2719cf0dc00e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c5b125928c1443f86967c0f7c56fcec', 'request_id': 'req-2fb710a0-1de6-4a4e-a8f6-2719cf0dc00e'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:21.422815 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-2fb710a0-1de6-4a4e-a8f6-2719cf0dc00e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c5b125928c1443f86967c0f7c56fcec', 'request_id': 'req-2fb710a0-1de6-4a4e-a8f6-2719cf0dc00e'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:21.429822 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-2fb710a0-1de6-4a4e-a8f6-2719cf0dc00e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c5b125928c1443f86967c0f7c56fcec', 'request_id': 'req-2fb710a0-1de6-4a4e-a8f6-2719cf0dc00e'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:21.440610 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-2fb710a0-1de6-4a4e-a8f6-2719cf0dc00e admin admin] The token cache is being invalidate because project 3c5b125928c1443f86967c0f7c56fcec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:51:21.440901 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-2fb710a0-1de6-4a4e-a8f6-2719cf0dc00e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:21.441537 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-2fb710a0-1de6-4a4e-a8f6-2719cf0dc00e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:51:21.443020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1219/2441] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:51:21 2022] DELETE /identity/v3/projects/3c5b125928c1443f86967c0f7c56fcec => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:51:25.379938 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccc17b1-27c8-4255-864a-d2d2b95e3874 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:51:25.381496 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-cccc17b1-27c8-4255-864a-d2d2b95e3874 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:25.457160 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccc17b1-27c8-4255-864a-d2d2b95e3874 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:51:25.504224 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a84321-f02d-4b99-a217-33e23018251c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:51:25.504747 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a84321-f02d-4b99-a217-33e23018251c service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:25.505022 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a84321-f02d-4b99-a217-33e23018251c service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:25.505287 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a84321-f02d-4b99-a217-33e23018251c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:25.507000 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-31a84321-f02d-4b99-a217-33e23018251c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:25.562882 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a84321-f02d-4b99-a217-33e23018251c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:51:25.564352 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a84321-f02d-4b99-a217-33e23018251c service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:51:25.573213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1223/2442] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:51:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:51:31.437660 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:51:32.867713 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0207f-6ddc-4be2-acde-bfee73c4f5af None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:32.868367 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0207f-6ddc-4be2-acde-bfee73c4f5af None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:32.868367 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0207f-6ddc-4be2-acde-bfee73c4f5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:32.887491 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-a4b0207f-6ddc-4be2-acde-bfee73c4f5af None None] MFA Rules not processed for user `047293a64a9f4857be1621e77094d367`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:51:32.889324 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-a4b0207f-6ddc-4be2-acde-bfee73c4f5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:32.903669 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1220/2443] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:51:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:51:34.592847 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafaad2-8692-4f16-9010-8cfd616cc608 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:51:34.593317 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafaad2-8692-4f16-9010-8cfd616cc608 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:51:34.593431 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafaad2-8692-4f16-9010-8cfd616cc608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:51:34.614743 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-8dafaad2-8692-4f16-9010-8cfd616cc608 None None] MFA Rules not processed for user `185b05ace78d4e369cb0354a0865fd52`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:51:34.615947 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-8dafaad2-8692-4f16-9010-8cfd616cc608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:51:34.640483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:51:34.676098 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1224/2444] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:51:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:51:42.882686 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=3 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:51:44.669282 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=12 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:52:02.690461 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed949909-0e49-40b2-8b19-6a910aca955d None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:02.690918 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed949909-0e49-40b2-8b19-6a910aca955d None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:02.691012 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed949909-0e49-40b2-8b19-6a910aca955d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:02.720174 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-ed949909-0e49-40b2-8b19-6a910aca955d None None] MFA Rules not processed for user `d0a13dbffb4a49508b7fb661cf84e803`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:52:02.721391 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-ed949909-0e49-40b2-8b19-6a910aca955d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:52:02.783208 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1221/2445] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Aug 17 23:52:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:52:02.795669 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dafaad2-8692-4f16-9010-8cfd616cc608 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:52:02.797095 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-8dafaad2-8692-4f16-9010-8cfd616cc608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:52:02.873938 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dafaad2-8692-4f16-9010-8cfd616cc608 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:52:02.878672 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ff377b-d770-42f3-9f2c-3439ed946211 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee422ea90557444fabdc9c292865b9b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:52:02.879269 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff377b-d770-42f3-9f2c-3439ed946211 service swift] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:02.879544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff377b-d770-42f3-9f2c-3439ed946211 service swift] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:02.879886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff377b-d770-42f3-9f2c-3439ed946211 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:02.894903 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ff377b-d770-42f3-9f2c-3439ed946211 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:52:02.896539 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ff377b-d770-42f3-9f2c-3439ed946211 service swift] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:52:02.904821 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1225/2446] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Aug 17 23:52:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 23:52:03.725681 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed949909-0e49-40b2-8b19-6a910aca955d None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:52:03.727153 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-ed949909-0e49-40b2-8b19-6a910aca955d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:52:03.864990 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed949909-0e49-40b2-8b19-6a910aca955d None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:52:03.871052 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10664c6c-e0c3-429f-9326-50e894d181a6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc57d911a56423e96c736bc96270ba8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:52:03.871839 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10664c6c-e0c3-429f-9326-50e894d181a6 service placement] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:03.872222 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10664c6c-e0c3-429f-9326-50e894d181a6 service placement] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:03.873553 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10664c6c-e0c3-429f-9326-50e894d181a6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:03.875398 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-10664c6c-e0c3-429f-9326-50e894d181a6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:52:03.888012 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10664c6c-e0c3-429f-9326-50e894d181a6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:52:03.889740 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10664c6c-e0c3-429f-9326-50e894d181a6 service placement] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:52:03.916526 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1222/2447] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:52:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:52:08.317763 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc34fec-fa50-4890-9d0d-c84de0a0aa56 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:08.318669 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc34fec-fa50-4890-9d0d-c84de0a0aa56 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:08.319230 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc34fec-fa50-4890-9d0d-c84de0a0aa56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:08.346708 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-cfc34fec-fa50-4890-9d0d-c84de0a0aa56 None None] MFA Rules not processed for user `21e06d9bb5a240459ef3298a990a42f2`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:52:08.348152 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-cfc34fec-fa50-4890-9d0d-c84de0a0aa56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:52:08.418728 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1226/2448] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:52:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:52:13.898127 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=73 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:52:18.404343 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=57 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:52:18.633061 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e3a05-a8a3-49fe-b0f5-1eece9d88399 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:18.633540 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e3a05-a8a3-49fe-b0f5-1eece9d88399 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:18.633649 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e3a05-a8a3-49fe-b0f5-1eece9d88399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:18.652843 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-dc0e3a05-a8a3-49fe-b0f5-1eece9d88399 None None] MFA Rules not processed for user `d0a13dbffb4a49508b7fb661cf84e803`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:52:18.654025 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-dc0e3a05-a8a3-49fe-b0f5-1eece9d88399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:52:18.661181 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1223/2449] 10.222.0.26 () {58 vars in 967 bytes} [Wed Aug 17 23:52:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:52:18.673147 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc34fec-fa50-4890-9d0d-c84de0a0aa56 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:52:18.685382 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc34fec-fa50-4890-9d0d-c84de0a0aa56 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:52:18.690407 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74de49-38dd-414c-bc08-d236bdae7e02 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee422ea90557444fabdc9c292865b9b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:52:18.695172 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74de49-38dd-414c-bc08-d236bdae7e02 service swift] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:18.695451 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74de49-38dd-414c-bc08-d236bdae7e02 service swift] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:18.695718 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74de49-38dd-414c-bc08-d236bdae7e02 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:18.711446 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74de49-38dd-414c-bc08-d236bdae7e02 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:52:18.713085 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74de49-38dd-414c-bc08-d236bdae7e02 service swift] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:52:18.721219 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1227/2450] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Aug 17 23:52:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 23:52:20.659800 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e3a05-a8a3-49fe-b0f5-1eece9d88399 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:52:20.661276 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-dc0e3a05-a8a3-49fe-b0f5-1eece9d88399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:52:20.725676 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e3a05-a8a3-49fe-b0f5-1eece9d88399 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:52:20.730193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef45fa8-cba3-4327-85a6-790238d2f9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:52:20.730819 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef45fa8-cba3-4327-85a6-790238d2f9f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:20.731097 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef45fa8-cba3-4327-85a6-790238d2f9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:20.731926 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef45fa8-cba3-4327-85a6-790238d2f9f8 admin admin] PATH_INFO: `/v3/users/185b05ace78d4e369cb0354a0865fd52` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:20.733076 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef45fa8-cba3-4327-85a6-790238d2f9f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=185b05ace78d4e369cb0354a0865fd52)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:52:20.734997 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef45fa8-cba3-4327-85a6-790238d2f9f8 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:52:20.779148 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-4ef45fa8-cba3-4327-85a6-790238d2f9f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '185b05ace78d4e369cb0354a0865fd52'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:52:20.779793 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-4ef45fa8-cba3-4327-85a6-790238d2f9f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '185b05ace78d4e369cb0354a0865fd52'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:52:20.780437 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-4ef45fa8-cba3-4327-85a6-790238d2f9f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '185b05ace78d4e369cb0354a0865fd52'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:52:20.787805 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-4ef45fa8-cba3-4327-85a6-790238d2f9f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '185b05ace78d4e369cb0354a0865fd52'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:52:20.795204 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-4ef45fa8-cba3-4327-85a6-790238d2f9f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '185b05ace78d4e369cb0354a0865fd52'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:52:20.811785 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1224/2451] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:52:20 2022] DELETE /identity/v3/users/185b05ace78d4e369cb0354a0865fd52 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:52:20.991580 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74de49-38dd-414c-bc08-d236bdae7e02 service swift] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:52:21.002691 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74de49-38dd-414c-bc08-d236bdae7e02 service swift] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:52:21.061912 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a685e-dd42-4327-8e13-483d565a3b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:52:21.062642 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a685e-dd42-4327-8e13-483d565a3b81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:21.062935 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a685e-dd42-4327-8e13-483d565a3b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:21.063191 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a685e-dd42-4327-8e13-483d565a3b81 admin admin] PATH_INFO: `/v3/projects/2ccde22a7c8a4a9bbfaab1dad9240f34` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:21.064339 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67a685e-dd42-4327-8e13-483d565a3b81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ccde22a7c8a4a9bbfaab1dad9240f34)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:52:21.066242 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67a685e-dd42-4327-8e13-483d565a3b81 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:52:21.126503 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-e67a685e-dd42-4327-8e13-483d565a3b81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ccde22a7c8a4a9bbfaab1dad9240f34', 'request_id': 'req-e67a685e-dd42-4327-8e13-483d565a3b81'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:52:21.133873 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-e67a685e-dd42-4327-8e13-483d565a3b81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ccde22a7c8a4a9bbfaab1dad9240f34', 'request_id': 'req-e67a685e-dd42-4327-8e13-483d565a3b81'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:52:21.146672 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-e67a685e-dd42-4327-8e13-483d565a3b81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ccde22a7c8a4a9bbfaab1dad9240f34', 'request_id': 'req-e67a685e-dd42-4327-8e13-483d565a3b81'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:52:21.159092 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-e67a685e-dd42-4327-8e13-483d565a3b81 admin admin] The token cache is being invalidate because project 2ccde22a7c8a4a9bbfaab1dad9240f34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:52:21.159305 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-e67a685e-dd42-4327-8e13-483d565a3b81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:52:21.159992 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-e67a685e-dd42-4327-8e13-483d565a3b81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:52:21.161777 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1228/2452] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:52:20 2022] DELETE /identity/v3/projects/2ccde22a7c8a4a9bbfaab1dad9240f34 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:52:21.166997 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f086833-2d04-455d-8cdf-85ad07d122b5 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:21.167281 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f086833-2d04-455d-8cdf-85ad07d122b5 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:21.167552 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f086833-2d04-455d-8cdf-85ad07d122b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:21.227668 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-6f086833-2d04-455d-8cdf-85ad07d122b5 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:52:21.228944 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-6f086833-2d04-455d-8cdf-85ad07d122b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:52:21.266833 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:52:21.289444 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1225/2453] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:52:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:52:21.297109 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e8b3a-b421-4dd7-893b-9d38583014ca None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:21.297394 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e8b3a-b421-4dd7-893b-9d38583014ca None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:21.297693 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e8b3a-b421-4dd7-893b-9d38583014ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:21.316631 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-ec8e8b3a-b421-4dd7-893b-9d38583014ca None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:52:21.317854 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-ec8e8b3a-b421-4dd7-893b-9d38583014ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:52:21.326107 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1229/2454] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:52:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:52:21.330102 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f086833-2d04-455d-8cdf-85ad07d122b5 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:52:21.339577 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f086833-2d04-455d-8cdf-85ad07d122b5 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:52:21.344786 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3596b86d-7ee6-4a85-b83c-76070f322372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:52:21.345373 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3596b86d-7ee6-4a85-b83c-76070f322372 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:21.345647 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3596b86d-7ee6-4a85-b83c-76070f322372 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:21.345915 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3596b86d-7ee6-4a85-b83c-76070f322372 admin admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:21.346557 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3596b86d-7ee6-4a85-b83c-76070f322372 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:52:21.346855 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3596b86d-7ee6-4a85-b83c-76070f322372 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:52:21.348464 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3596b86d-7ee6-4a85-b83c-76070f322372 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:52:21.359985 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1226/2455] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:52:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:52:21.364144 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e8b3a-b421-4dd7-893b-9d38583014ca None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:52:21.373714 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e8b3a-b421-4dd7-893b-9d38583014ca None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:52:21.378246 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51f613-5f4a-4c22-a027-1beff23c8051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:52:21.378826 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51f613-5f4a-4c22-a027-1beff23c8051 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:21.379099 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51f613-5f4a-4c22-a027-1beff23c8051 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:21.379363 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51f613-5f4a-4c22-a027-1beff23c8051 admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:21.380042 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f51f613-5f4a-4c22-a027-1beff23c8051 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:52:21.381914 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f51f613-5f4a-4c22-a027-1beff23c8051 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:52:21.409061 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1230/2456] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:52:21 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:52:21.413502 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3596b86d-7ee6-4a85-b83c-76070f322372 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:52:21.423505 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3596b86d-7ee6-4a85-b83c-76070f322372 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:52:21.478559 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef3ed22-cc51-43c7-bf19-c9f4dd019ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:52:21.479156 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3ed22-cc51-43c7-bf19-c9f4dd019ba6 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:21.479433 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3ed22-cc51-43c7-bf19-c9f4dd019ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:21.479700 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3ed22-cc51-43c7-bf19-c9f4dd019ba6 admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:21.480366 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef3ed22-cc51-43c7-bf19-c9f4dd019ba6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:52:21.482303 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef3ed22-cc51-43c7-bf19-c9f4dd019ba6 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:52:21.502237 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1227/2457] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:52:21 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:52:21.506401 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51f613-5f4a-4c22-a027-1beff23c8051 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:52:21.517310 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51f613-5f4a-4c22-a027-1beff23c8051 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:52:21.521897 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f79ef58-8763-443e-8552-91a7616581dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:52:21.522487 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79ef58-8763-443e-8552-91a7616581dc admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:21.522767 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79ef58-8763-443e-8552-91a7616581dc admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:21.523039 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79ef58-8763-443e-8552-91a7616581dc admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:21.523682 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f79ef58-8763-443e-8552-91a7616581dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:52:21.525320 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f79ef58-8763-443e-8552-91a7616581dc admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:52:21.535217 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1231/2458] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:52:21.539325 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef3ed22-cc51-43c7-bf19-c9f4dd019ba6 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:52:21.549034 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef3ed22-cc51-43c7-bf19-c9f4dd019ba6 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:52:21.553526 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609661f-bc9e-4c29-a8c7-1a1747b49ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:52:21.554106 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609661f-bc9e-4c29-a8c7-1a1747b49ab5 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:21.554382 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609661f-bc9e-4c29-a8c7-1a1747b49ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:21.554654 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609661f-bc9e-4c29-a8c7-1a1747b49ab5 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:21.555286 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3609661f-bc9e-4c29-a8c7-1a1747b49ab5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:52:21.556895 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3609661f-bc9e-4c29-a8c7-1a1747b49ab5 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:52:21.567568 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1228/2459] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:52:21.571343 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f79ef58-8763-443e-8552-91a7616581dc admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:52:21.582414 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f79ef58-8763-443e-8552-91a7616581dc admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:52:21.587161 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46615b3-c566-4b06-aa10-6ed2ad3dde64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:52:21.587834 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46615b3-c566-4b06-aa10-6ed2ad3dde64 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:21.588152 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46615b3-c566-4b06-aa10-6ed2ad3dde64 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:21.588421 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46615b3-c566-4b06-aa10-6ed2ad3dde64 admin admin] PATH_INFO: `/v3/projects/7d90170c7ffc45b091d8f4b41dd751a0/users/38d763bd1cd54f76bb33cae29abf8ab1/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:21.602981 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46615b3-c566-4b06-aa10-6ed2ad3dde64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d90170c7ffc45b091d8f4b41dd751a0, user_id=38d763bd1cd54f76bb33cae29abf8ab1, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:52:21.605665 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46615b3-c566-4b06-aa10-6ed2ad3dde64 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:52:21.619671 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1232/2460] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:52:21 2022] PUT /identity/v3/projects/7d90170c7ffc45b091d8f4b41dd751a0/users/38d763bd1cd54f76bb33cae29abf8ab1/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:52:21.630334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73c847-258a-46e0-a19a-474137b04678 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:21.630693 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73c847-258a-46e0-a19a-474137b04678 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:21.631030 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73c847-258a-46e0-a19a-474137b04678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:21.661491 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-ec73c847-258a-46e0-a19a-474137b04678 None None] MFA Rules not processed for user `38d763bd1cd54f76bb33cae29abf8ab1`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:52:21.663407 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-ec73c847-258a-46e0-a19a-474137b04678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:52:21.737068 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1229/2461] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:52:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:52:21.748403 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46615b3-c566-4b06-aa10-6ed2ad3dde64 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:52:21.750116 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-e46615b3-c566-4b06-aa10-6ed2ad3dde64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:52:21.768829 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:52:21.829579 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46615b3-c566-4b06-aa10-6ed2ad3dde64 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:52:21.833169 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb03a1-9db1-47e0-a633-0fe348a75f13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:52:21.833762 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb03a1-9db1-47e0-a633-0fe348a75f13 service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:21.834036 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb03a1-9db1-47e0-a633-0fe348a75f13 service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:21.834303 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb03a1-9db1-47e0-a633-0fe348a75f13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:21.842331 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb03a1-9db1-47e0-a633-0fe348a75f13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:52:21.843750 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb03a1-9db1-47e0-a633-0fe348a75f13 service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:52:21.852801 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1233/2462] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:52:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:52:24.395148 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec73c847-258a-46e0-a19a-474137b04678 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:52:24.439687 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:52:24.465099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec73c847-258a-46e0-a19a-474137b04678 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:52:24.469818 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2535f-aa50-4af1-91a6-829759d4c965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:52:24.470481 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2535f-aa50-4af1-91a6-829759d4c965 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:24.470764 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2535f-aa50-4af1-91a6-829759d4c965 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:24.471158 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2535f-aa50-4af1-91a6-829759d4c965 admin admin] PATH_INFO: `/v3/users/38d763bd1cd54f76bb33cae29abf8ab1` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:24.472336 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca2535f-aa50-4af1-91a6-829759d4c965 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38d763bd1cd54f76bb33cae29abf8ab1)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:52:24.474284 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca2535f-aa50-4af1-91a6-829759d4c965 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:52:24.522379 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-fca2535f-aa50-4af1-91a6-829759d4c965 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38d763bd1cd54f76bb33cae29abf8ab1'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:52:24.523035 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-fca2535f-aa50-4af1-91a6-829759d4c965 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38d763bd1cd54f76bb33cae29abf8ab1'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:52:24.523647 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-fca2535f-aa50-4af1-91a6-829759d4c965 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38d763bd1cd54f76bb33cae29abf8ab1'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:52:24.531273 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-fca2535f-aa50-4af1-91a6-829759d4c965 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38d763bd1cd54f76bb33cae29abf8ab1'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:52:24.538594 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-fca2535f-aa50-4af1-91a6-829759d4c965 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38d763bd1cd54f76bb33cae29abf8ab1'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:52:24.556425 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1230/2463] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:52:24 2022] DELETE /identity/v3/users/38d763bd1cd54f76bb33cae29abf8ab1 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:52:24.561421 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3314cc5-9d74-457d-b442-700497309ff3 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:24.561886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3314cc5-9d74-457d-b442-700497309ff3 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:24.562052 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3314cc5-9d74-457d-b442-700497309ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:24.581761 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-c3314cc5-9d74-457d-b442-700497309ff3 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:52:24.583201 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-c3314cc5-9d74-457d-b442-700497309ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:52:24.655424 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1234/2464] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:52:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:52:24.665123 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2535f-aa50-4af1-91a6-829759d4c965 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:52:24.666639 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-fca2535f-aa50-4af1-91a6-829759d4c965 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:52:24.743146 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2535f-aa50-4af1-91a6-829759d4c965 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:52:24.747318 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c514b6-0901-4dfb-99d3-e4f03be88cb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:52:24.747920 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c514b6-0901-4dfb-99d3-e4f03be88cb0 service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:24.748227 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c514b6-0901-4dfb-99d3-e4f03be88cb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:24.748498 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c514b6-0901-4dfb-99d3-e4f03be88cb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:24.757286 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c514b6-0901-4dfb-99d3-e4f03be88cb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:52:24.758717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c514b6-0901-4dfb-99d3-e4f03be88cb0 service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:52:24.768527 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1231/2465] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:52:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:52:25.047363 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3314cc5-9d74-457d-b442-700497309ff3 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:52:25.048865 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-c3314cc5-9d74-457d-b442-700497309ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:52:25.062481 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3314cc5-9d74-457d-b442-700497309ff3 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:52:25.067115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb2e8e-8923-4991-b31e-ce7aef26647c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:52:25.067749 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb2e8e-8923-4991-b31e-ce7aef26647c admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:25.068064 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb2e8e-8923-4991-b31e-ce7aef26647c admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:25.068339 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb2e8e-8923-4991-b31e-ce7aef26647c admin admin] PATH_INFO: `/v3/projects/7d90170c7ffc45b091d8f4b41dd751a0` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:25.069458 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfb2e8e-8923-4991-b31e-ce7aef26647c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d90170c7ffc45b091d8f4b41dd751a0)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:52:25.071373 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfb2e8e-8923-4991-b31e-ce7aef26647c admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:52:25.135954 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-8bfb2e8e-8923-4991-b31e-ce7aef26647c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d90170c7ffc45b091d8f4b41dd751a0', 'request_id': 'req-8bfb2e8e-8923-4991-b31e-ce7aef26647c'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:52:25.143221 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-8bfb2e8e-8923-4991-b31e-ce7aef26647c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d90170c7ffc45b091d8f4b41dd751a0', 'request_id': 'req-8bfb2e8e-8923-4991-b31e-ce7aef26647c'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:52:25.152560 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-8bfb2e8e-8923-4991-b31e-ce7aef26647c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d90170c7ffc45b091d8f4b41dd751a0', 'request_id': 'req-8bfb2e8e-8923-4991-b31e-ce7aef26647c'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:52:25.167457 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-8bfb2e8e-8923-4991-b31e-ce7aef26647c admin admin] The token cache is being invalidate because project 7d90170c7ffc45b091d8f4b41dd751a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:52:25.167873 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-8bfb2e8e-8923-4991-b31e-ce7aef26647c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:52:25.169338 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-8bfb2e8e-8923-4991-b31e-ce7aef26647c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:52:25.172032 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1235/2466] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:52:25 2022] DELETE /identity/v3/projects/7d90170c7ffc45b091d8f4b41dd751a0 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:52:25.176291 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca8ddf-9288-4abf-a93c-0030a5f973e2 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:25.176672 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca8ddf-9288-4abf-a93c-0030a5f973e2 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:25.177046 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca8ddf-9288-4abf-a93c-0030a5f973e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:25.203143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-c7ca8ddf-9288-4abf-a93c-0030a5f973e2 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:52:25.203903 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-c7ca8ddf-9288-4abf-a93c-0030a5f973e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:52:25.251271 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:52:25.294475 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1232/2467] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:52:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:52:25.298342 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96bab0c-aa0a-4500-9113-f2348725e195 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:25.298644 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96bab0c-aa0a-4500-9113-f2348725e195 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:25.298935 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96bab0c-aa0a-4500-9113-f2348725e195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:25.318554 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-f96bab0c-aa0a-4500-9113-f2348725e195 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:52:25.319741 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-f96bab0c-aa0a-4500-9113-f2348725e195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:52:25.327712 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1236/2468] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:52:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:52:25.331833 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca8ddf-9288-4abf-a93c-0030a5f973e2 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:52:25.342191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca8ddf-9288-4abf-a93c-0030a5f973e2 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:52:25.346777 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4212390-9c93-46ae-8be0-52666dbf393f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:52:25.347370 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4212390-9c93-46ae-8be0-52666dbf393f admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:25.347649 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4212390-9c93-46ae-8be0-52666dbf393f admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:25.347924 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4212390-9c93-46ae-8be0-52666dbf393f admin admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:25.348596 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4212390-9c93-46ae-8be0-52666dbf393f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:52:25.348899 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4212390-9c93-46ae-8be0-52666dbf393f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:52:25.350485 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4212390-9c93-46ae-8be0-52666dbf393f admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:52:25.362467 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1233/2469] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:52:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:52:25.366604 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96bab0c-aa0a-4500-9113-f2348725e195 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:52:25.376328 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96bab0c-aa0a-4500-9113-f2348725e195 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:52:25.380949 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2399c5-7c9a-472c-99a3-782b83f87725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:52:25.381535 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2399c5-7c9a-472c-99a3-782b83f87725 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:25.381813 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2399c5-7c9a-472c-99a3-782b83f87725 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:25.382080 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2399c5-7c9a-472c-99a3-782b83f87725 admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:25.382699 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2399c5-7c9a-472c-99a3-782b83f87725 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:52:25.384633 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2399c5-7c9a-472c-99a3-782b83f87725 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:52:25.396027 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:52:25.413466 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1237/2470] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:52:25 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:52:25.417494 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4212390-9c93-46ae-8be0-52666dbf393f admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:52:25.428234 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4212390-9c93-46ae-8be0-52666dbf393f admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:52:25.485253 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2340bf28-272d-4ffc-9a05-32028f1de73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:52:25.485943 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340bf28-272d-4ffc-9a05-32028f1de73b admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:25.486220 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340bf28-272d-4ffc-9a05-32028f1de73b admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:25.486491 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340bf28-272d-4ffc-9a05-32028f1de73b admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:25.487140 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2340bf28-272d-4ffc-9a05-32028f1de73b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:52:25.489240 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2340bf28-272d-4ffc-9a05-32028f1de73b admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:52:25.510406 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1234/2471] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:52:25 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:52:25.515263 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2399c5-7c9a-472c-99a3-782b83f87725 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:52:25.525903 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2399c5-7c9a-472c-99a3-782b83f87725 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:52:25.530670 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114b418-2873-4134-a43f-f5d583d23c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:52:25.531254 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114b418-2873-4134-a43f-f5d583d23c15 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:25.531530 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114b418-2873-4134-a43f-f5d583d23c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:25.531823 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114b418-2873-4134-a43f-f5d583d23c15 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:25.532483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c114b418-2873-4134-a43f-f5d583d23c15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:52:25.534067 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c114b418-2873-4134-a43f-f5d583d23c15 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:52:25.544473 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1238/2472] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:52:25.548343 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2340bf28-272d-4ffc-9a05-32028f1de73b admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:52:25.558206 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2340bf28-272d-4ffc-9a05-32028f1de73b admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:52:25.563378 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf90a2-142e-4711-acb1-4c077c8cdfee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:52:25.563994 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf90a2-142e-4711-acb1-4c077c8cdfee admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:25.564278 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf90a2-142e-4711-acb1-4c077c8cdfee admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:25.564553 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf90a2-142e-4711-acb1-4c077c8cdfee admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:25.565218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf90a2-142e-4711-acb1-4c077c8cdfee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:52:25.566804 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf90a2-142e-4711-acb1-4c077c8cdfee admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:52:25.577226 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1235/2473] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:52:25.581458 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114b418-2873-4134-a43f-f5d583d23c15 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:52:25.591381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114b418-2873-4134-a43f-f5d583d23c15 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:52:25.595862 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d65017-0d4e-4348-9162-bf6b0636654d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:52:25.596518 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d65017-0d4e-4348-9162-bf6b0636654d admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:25.596794 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d65017-0d4e-4348-9162-bf6b0636654d admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:25.597058 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d65017-0d4e-4348-9162-bf6b0636654d admin admin] PATH_INFO: `/v3/projects/190eae26a0d5487882d1c59c337f0db4/users/64fdf46f185c48558ad1a637898745a9/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:25.611522 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d65017-0d4e-4348-9162-bf6b0636654d admin admin] RBAC: Authorizing `identity:create_grant(project_id=190eae26a0d5487882d1c59c337f0db4, user_id=64fdf46f185c48558ad1a637898745a9, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:52:25.614195 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d65017-0d4e-4348-9162-bf6b0636654d admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:52:25.627076 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1239/2474] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:52:25 2022] PUT /identity/v3/projects/190eae26a0d5487882d1c59c337f0db4/users/64fdf46f185c48558ad1a637898745a9/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:52:25.632088 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9dc955-d5f5-4866-a9b2-b3d19b94d895 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:25.632371 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9dc955-d5f5-4866-a9b2-b3d19b94d895 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:25.632645 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9dc955-d5f5-4866-a9b2-b3d19b94d895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:25.650847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-2d9dc955-d5f5-4866-a9b2-b3d19b94d895 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:52:25.652089 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-2d9dc955-d5f5-4866-a9b2-b3d19b94d895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:52:25.708983 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1236/2475] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:52:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:52:25.718711 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d65017-0d4e-4348-9162-bf6b0636654d admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:52:25.720408 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-99d65017-0d4e-4348-9162-bf6b0636654d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:52:25.776377 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d65017-0d4e-4348-9162-bf6b0636654d admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:52:25.780065 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6fe85-c805-491f-bf73-bcd81abf6183 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:52:25.780683 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6fe85-c805-491f-bf73-bcd81abf6183 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:25.780958 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6fe85-c805-491f-bf73-bcd81abf6183 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:25.781224 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6fe85-c805-491f-bf73-bcd81abf6183 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:25.790184 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd6fe85-c805-491f-bf73-bcd81abf6183 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:52:25.791637 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd6fe85-c805-491f-bf73-bcd81abf6183 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:52:25.801417 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1240/2476] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:52:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:52:26.508824 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35893df-f194-4e98-8438-f17b9c8d4d8c None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:26.509309 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35893df-f194-4e98-8438-f17b9c8d4d8c None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:26.509408 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35893df-f194-4e98-8438-f17b9c8d4d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:26.539113 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-f35893df-f194-4e98-8438-f17b9c8d4d8c None None] MFA Rules not processed for user `64fdf46f185c48558ad1a637898745a9`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:52:26.540365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-f35893df-f194-4e98-8438-f17b9c8d4d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:52:26.572671 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:52:26.608947 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1237/2477] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:52:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:52:26.619642 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6fe85-c805-491f-bf73-bcd81abf6183 service neutron] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:52:26.621195 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-ddd6fe85-c805-491f-bf73-bcd81abf6183 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:52:26.680090 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6fe85-c805-491f-bf73-bcd81abf6183 service neutron] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:52:26.683678 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07e66c-8240-499e-9be2-23796e9d331b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:52:26.684871 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07e66c-8240-499e-9be2-23796e9d331b service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:26.685151 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07e66c-8240-499e-9be2-23796e9d331b service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:26.685424 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07e66c-8240-499e-9be2-23796e9d331b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:26.694578 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c07e66c-8240-499e-9be2-23796e9d331b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:52:26.696075 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c07e66c-8240-499e-9be2-23796e9d331b service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:52:26.706603 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1241/2478] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:52:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:52:29.112048 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b38abe-7a4a-40e3-9f2b-9ca9d5ed1bc4 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:29.112555 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b38abe-7a4a-40e3-9f2b-9ca9d5ed1bc4 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:29.112555 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b38abe-7a4a-40e3-9f2b-9ca9d5ed1bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:29.130773 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-12b38abe-7a4a-40e3-9f2b-9ca9d5ed1bc4 None None] MFA Rules not processed for user `64fdf46f185c48558ad1a637898745a9`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:52:29.131952 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-12b38abe-7a4a-40e3-9f2b-9ca9d5ed1bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:52:29.138636 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1238/2479] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:52:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:52:29.877470 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5d8ea-2b2b-4964-b128-543f4e1bca30 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:29.877470 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5d8ea-2b2b-4964-b128-543f4e1bca30 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:29.878075 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5d8ea-2b2b-4964-b128-543f4e1bca30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:29.897430 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-26e5d8ea-2b2b-4964-b128-543f4e1bca30 None None] MFA Rules not processed for user `d0a13dbffb4a49508b7fb661cf84e803`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:52:29.898646 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-26e5d8ea-2b2b-4964-b128-543f4e1bca30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:52:29.965513 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1242/2480] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Aug 17 23:52:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:52:29.974205 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b38abe-7a4a-40e3-9f2b-9ca9d5ed1bc4 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:52:29.975862 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-12b38abe-7a4a-40e3-9f2b-9ca9d5ed1bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:52:30.067269 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b38abe-7a4a-40e3-9f2b-9ca9d5ed1bc4 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:52:30.073160 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36714cbf-c4d0-4137-b7c5-14f57e8adafd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee422ea90557444fabdc9c292865b9b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:52:30.073845 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36714cbf-c4d0-4137-b7c5-14f57e8adafd service swift] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:30.074147 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36714cbf-c4d0-4137-b7c5-14f57e8adafd service swift] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:30.074428 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36714cbf-c4d0-4137-b7c5-14f57e8adafd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:30.087914 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36714cbf-c4d0-4137-b7c5-14f57e8adafd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:52:30.089584 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36714cbf-c4d0-4137-b7c5-14f57e8adafd service swift] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:52:30.098440 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1239/2481] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Aug 17 23:52:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 23:52:39.952228 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=95,keystone:INSERT=2 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:52:40.094776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=46 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:52:50.165220 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b61e8a-421e-40a7-b723-8e8e8111a8ea None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:50.165736 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b61e8a-421e-40a7-b723-8e8e8111a8ea None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:50.165843 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b61e8a-421e-40a7-b723-8e8e8111a8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:50.185134 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-e2b61e8a-421e-40a7-b723-8e8e8111a8ea None None] MFA Rules not processed for user `d0a13dbffb4a49508b7fb661cf84e803`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:52:50.186328 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-e2b61e8a-421e-40a7-b723-8e8e8111a8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:52:50.193491 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1243/2482] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Aug 17 23:52:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:52:50.205830 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36714cbf-c4d0-4137-b7c5-14f57e8adafd service swift] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:52:50.216385 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36714cbf-c4d0-4137-b7c5-14f57e8adafd service swift] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:52:50.222025 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79508af8-54b2-4e08-a0e6-a5c2747255fa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee422ea90557444fabdc9c292865b9b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:52:50.222643 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79508af8-54b2-4e08-a0e6-a5c2747255fa service swift] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:52:50.222928 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79508af8-54b2-4e08-a0e6-a5c2747255fa service swift] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:52:50.223203 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79508af8-54b2-4e08-a0e6-a5c2747255fa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:52:50.232176 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79508af8-54b2-4e08-a0e6-a5c2747255fa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:52:50.233778 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79508af8-54b2-4e08-a0e6-a5c2747255fa service swift] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:52:50.242480 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1240/2483] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Aug 17 23:52:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 23:53:00.180650 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=3 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:53:00.239070 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=6 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:53:13.157316 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b61e8a-421e-40a7-b723-8e8e8111a8ea None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:53:13.158736 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-e2b61e8a-421e-40a7-b723-8e8e8111a8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:53:13.171320 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b61e8a-421e-40a7-b723-8e8e8111a8ea None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:53:13.175506 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfce738-f3de-4805-bab2-3bf6347a3b1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:53:13.176136 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfce738-f3de-4805-bab2-3bf6347a3b1f service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:53:13.176417 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfce738-f3de-4805-bab2-3bf6347a3b1f service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:53:13.176687 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfce738-f3de-4805-bab2-3bf6347a3b1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:53:13.178360 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-7dfce738-f3de-4805-bab2-3bf6347a3b1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:53:13.259078 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfce738-f3de-4805-bab2-3bf6347a3b1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:53:13.260559 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfce738-f3de-4805-bab2-3bf6347a3b1f service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:53:13.281007 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1244/2484] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:53:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:53:13.848916 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79508af8-54b2-4e08-a0e6-a5c2747255fa service swift] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:53:13.850338 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-79508af8-54b2-4e08-a0e6-a5c2747255fa service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:53:13.862876 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79508af8-54b2-4e08-a0e6-a5c2747255fa service swift] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:53:13.866415 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f718f34-df98-4495-b582-a263a617af50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:53:13.867005 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f718f34-df98-4495-b582-a263a617af50 service cinder] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:53:13.867284 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f718f34-df98-4495-b582-a263a617af50 service cinder] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:53:13.867554 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f718f34-df98-4495-b582-a263a617af50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:53:13.869254 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-7f718f34-df98-4495-b582-a263a617af50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:53:13.934800 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f718f34-df98-4495-b582-a263a617af50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:53:13.936188 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f718f34-df98-4495-b582-a263a617af50 service cinder] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:53:13.945727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1241/2485] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:53:13 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:53:14.151822 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfce738-f3de-4805-bab2-3bf6347a3b1f service neutron] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:53:14.160311 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfce738-f3de-4805-bab2-3bf6347a3b1f service neutron] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:53:14.164153 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd36e5d-2083-46e2-b974-8368d5d3d5c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:53:14.164748 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd36e5d-2083-46e2-b974-8368d5d3d5c0 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:53:14.165027 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd36e5d-2083-46e2-b974-8368d5d3d5c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:53:14.165297 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd36e5d-2083-46e2-b974-8368d5d3d5c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:53:14.166993 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-3fd36e5d-2083-46e2-b974-8368d5d3d5c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:53:14.226251 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd36e5d-2083-46e2-b974-8368d5d3d5c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:53:14.227670 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd36e5d-2083-46e2-b974-8368d5d3d5c0 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:53:14.236826 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1245/2486] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:53:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:53:15.696523 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f718f34-df98-4495-b582-a263a617af50 service cinder] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:53:15.705973 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f718f34-df98-4495-b582-a263a617af50 service cinder] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:53:15.709716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c431a2-1221-456f-baad-97453f3002dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:53:15.710329 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c431a2-1221-456f-baad-97453f3002dc service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:53:15.710608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c431a2-1221-456f-baad-97453f3002dc service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:53:15.710887 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c431a2-1221-456f-baad-97453f3002dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:53:15.712671 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-a8c431a2-1221-456f-baad-97453f3002dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:53:15.726157 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c431a2-1221-456f-baad-97453f3002dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:53:15.727601 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c431a2-1221-456f-baad-97453f3002dc service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:53:15.739289 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1242/2487] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:53:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:53:19.877132 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d70a22-1cc7-43a3-a0cc-e2a3b9c8f551 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:53:19.877633 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d70a22-1cc7-43a3-a0cc-e2a3b9c8f551 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:53:19.877738 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d70a22-1cc7-43a3-a0cc-e2a3b9c8f551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:53:19.896872 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-b8d70a22-1cc7-43a3-a0cc-e2a3b9c8f551 None None] MFA Rules not processed for user `d0a13dbffb4a49508b7fb661cf84e803`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:53:19.898092 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-b8d70a22-1cc7-43a3-a0cc-e2a3b9c8f551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:53:19.905312 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1246/2488] 10.222.0.26 () {58 vars in 967 bytes} [Wed Aug 17 23:53:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:53:19.930086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c431a2-1221-456f-baad-97453f3002dc service neutron] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:53:19.941278 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c431a2-1221-456f-baad-97453f3002dc service neutron] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:53:19.946716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd779fa-965b-4ae3-ace8-26787e826347 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee422ea90557444fabdc9c292865b9b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:53:19.947324 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd779fa-965b-4ae3-ace8-26787e826347 service swift] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:53:19.947605 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd779fa-965b-4ae3-ace8-26787e826347 service swift] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:53:19.947901 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd779fa-965b-4ae3-ace8-26787e826347 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:53:19.956357 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd779fa-965b-4ae3-ace8-26787e826347 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:53:19.957983 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd779fa-965b-4ae3-ace8-26787e826347 service swift] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:53:19.966592 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1243/2489] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Aug 17 23:53:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 17 23:53:26.023134 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa807f29-9dd2-4edb-b08c-5dbfe605d2a1 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:53:26.023611 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa807f29-9dd2-4edb-b08c-5dbfe605d2a1 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:53:26.023611 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa807f29-9dd2-4edb-b08c-5dbfe605d2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:53:26.042402 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-aa807f29-9dd2-4edb-b08c-5dbfe605d2a1 None None] MFA Rules not processed for user `cf6e346e96144628ac25c428be38870a`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:53:26.043604 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-aa807f29-9dd2-4edb-b08c-5dbfe605d2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:53:26.050966 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1247/2490] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:53:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:53:29.963115 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=40 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:53:32.643226 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd779fa-965b-4ae3-ace8-26787e826347 service swift] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:53:32.653192 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd779fa-965b-4ae3-ace8-26787e826347 service swift] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:53:32.657931 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fda763-04bd-4292-9cb4-cd78e11f7c3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:53:32.658625 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fda763-04bd-4292-9cb4-cd78e11f7c3e service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:53:32.658910 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fda763-04bd-4292-9cb4-cd78e11f7c3e service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:53:32.659180 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fda763-04bd-4292-9cb4-cd78e11f7c3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:53:32.661234 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-b7fda763-04bd-4292-9cb4-cd78e11f7c3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:53:32.677587 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fda763-04bd-4292-9cb4-cd78e11f7c3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:53:32.679020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fda763-04bd-4292-9cb4-cd78e11f7c3e service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:53:32.690351 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1244/2491] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:53:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:53:36.037654 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=77 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:53:40.100217 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa807f29-9dd2-4edb-b08c-5dbfe605d2a1 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:53:40.101716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-aa807f29-9dd2-4edb-b08c-5dbfe605d2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:53:40.121923 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa807f29-9dd2-4edb-b08c-5dbfe605d2a1 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:53:40.127886 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782879e-415f-4262-8c2e-00f3e0ae6bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:53:40.128546 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782879e-415f-4262-8c2e-00f3e0ae6bab admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:53:40.128818 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782879e-415f-4262-8c2e-00f3e0ae6bab admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:53:40.129112 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782879e-415f-4262-8c2e-00f3e0ae6bab admin admin] PATH_INFO: `/v3/users/ce1b5129d46345ef99c69e95f9824c45` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:53:40.130238 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6782879e-415f-4262-8c2e-00f3e0ae6bab admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce1b5129d46345ef99c69e95f9824c45)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:53:40.132239 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6782879e-415f-4262-8c2e-00f3e0ae6bab admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:53:40.178487 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-6782879e-415f-4262-8c2e-00f3e0ae6bab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce1b5129d46345ef99c69e95f9824c45'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:53:40.179135 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-6782879e-415f-4262-8c2e-00f3e0ae6bab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce1b5129d46345ef99c69e95f9824c45'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:53:40.191147 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-6782879e-415f-4262-8c2e-00f3e0ae6bab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce1b5129d46345ef99c69e95f9824c45'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:53:40.198029 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-6782879e-415f-4262-8c2e-00f3e0ae6bab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce1b5129d46345ef99c69e95f9824c45'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:53:40.198624 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-6782879e-415f-4262-8c2e-00f3e0ae6bab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce1b5129d46345ef99c69e95f9824c45'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:53:40.211258 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1248/2492] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:53:40 2022] DELETE /identity/v3/users/ce1b5129d46345ef99c69e95f9824c45 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:53:40.380147 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fda763-04bd-4292-9cb4-cd78e11f7c3e service neutron] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:53:40.396212 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fda763-04bd-4292-9cb4-cd78e11f7c3e service neutron] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:53:40.453986 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7d8db-d581-4048-9634-a268aadfd142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:53:40.454623 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7d8db-d581-4048-9634-a268aadfd142 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:53:40.454908 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7d8db-d581-4048-9634-a268aadfd142 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:53:40.455187 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7d8db-d581-4048-9634-a268aadfd142 admin admin] PATH_INFO: `/v3/projects/aac208652b8247049fa6a0bc9aa991bf` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:53:40.456355 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f7d8db-d581-4048-9634-a268aadfd142 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aac208652b8247049fa6a0bc9aa991bf)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:53:40.458271 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f7d8db-d581-4048-9634-a268aadfd142 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:53:40.506434 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-40f7d8db-d581-4048-9634-a268aadfd142 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aac208652b8247049fa6a0bc9aa991bf', 'request_id': 'req-40f7d8db-d581-4048-9634-a268aadfd142'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:53:40.511861 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-40f7d8db-d581-4048-9634-a268aadfd142 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aac208652b8247049fa6a0bc9aa991bf', 'request_id': 'req-40f7d8db-d581-4048-9634-a268aadfd142'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:53:40.518994 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-40f7d8db-d581-4048-9634-a268aadfd142 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aac208652b8247049fa6a0bc9aa991bf', 'request_id': 'req-40f7d8db-d581-4048-9634-a268aadfd142'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:53:40.529721 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-40f7d8db-d581-4048-9634-a268aadfd142 admin admin] The token cache is being invalidate because project aac208652b8247049fa6a0bc9aa991bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:53:40.530013 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-40f7d8db-d581-4048-9634-a268aadfd142 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:53:40.530640 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-40f7d8db-d581-4048-9634-a268aadfd142 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:53:40.532193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1245/2493] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:53:40 2022] DELETE /identity/v3/projects/aac208652b8247049fa6a0bc9aa991bf => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:53:40.536271 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782879e-415f-4262-8c2e-00f3e0ae6bab admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:53:40.537673 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-6782879e-415f-4262-8c2e-00f3e0ae6bab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:53:40.551522 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782879e-415f-4262-8c2e-00f3e0ae6bab admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:53:40.606783 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca720ac-77e2-4d7a-be88-25ca8f672667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:53:40.607345 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca720ac-77e2-4d7a-be88-25ca8f672667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:53:40.607622 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca720ac-77e2-4d7a-be88-25ca8f672667 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:53:40.607890 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca720ac-77e2-4d7a-be88-25ca8f672667 admin admin] PATH_INFO: `/v3/users/963f8c6c9f6d4b1aa076802c21021ae8` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:53:40.609040 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca720ac-77e2-4d7a-be88-25ca8f672667 admin admin] RBAC: Authorizing `identity:delete_user(user_id=963f8c6c9f6d4b1aa076802c21021ae8)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:53:40.611478 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca720ac-77e2-4d7a-be88-25ca8f672667 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:53:40.655941 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-7ca720ac-77e2-4d7a-be88-25ca8f672667 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '963f8c6c9f6d4b1aa076802c21021ae8'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:53:40.656720 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-7ca720ac-77e2-4d7a-be88-25ca8f672667 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '963f8c6c9f6d4b1aa076802c21021ae8'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:53:40.668671 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-7ca720ac-77e2-4d7a-be88-25ca8f672667 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '963f8c6c9f6d4b1aa076802c21021ae8'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:53:40.676229 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-7ca720ac-77e2-4d7a-be88-25ca8f672667 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '963f8c6c9f6d4b1aa076802c21021ae8'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:53:40.676853 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-7ca720ac-77e2-4d7a-be88-25ca8f672667 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '963f8c6c9f6d4b1aa076802c21021ae8'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:53:40.678847 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:53:40.691452 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1249/2494] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:53:40 2022] DELETE /identity/v3/users/963f8c6c9f6d4b1aa076802c21021ae8 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:53:40.873703 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7d8db-d581-4048-9634-a268aadfd142 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:53:40.875253 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-40f7d8db-d581-4048-9634-a268aadfd142 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:53:40.888056 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7d8db-d581-4048-9634-a268aadfd142 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:53:40.955160 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3eb03-5b1c-4286-9466-5a82c432a7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:53:40.956048 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3eb03-5b1c-4286-9466-5a82c432a7e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:53:40.956626 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3eb03-5b1c-4286-9466-5a82c432a7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:53:40.956901 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3eb03-5b1c-4286-9466-5a82c432a7e2 admin admin] PATH_INFO: `/v3/projects/25139a07847c44f89448109ec8fadd56` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:53:40.958026 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a3eb03-5b1c-4286-9466-5a82c432a7e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25139a07847c44f89448109ec8fadd56)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:53:40.960126 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a3eb03-5b1c-4286-9466-5a82c432a7e2 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:53:40.992563 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:53:41.034685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-d2a3eb03-5b1c-4286-9466-5a82c432a7e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25139a07847c44f89448109ec8fadd56', 'request_id': 'req-d2a3eb03-5b1c-4286-9466-5a82c432a7e2'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:53:41.041456 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-d2a3eb03-5b1c-4286-9466-5a82c432a7e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25139a07847c44f89448109ec8fadd56', 'request_id': 'req-d2a3eb03-5b1c-4286-9466-5a82c432a7e2'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:53:41.050702 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-d2a3eb03-5b1c-4286-9466-5a82c432a7e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25139a07847c44f89448109ec8fadd56', 'request_id': 'req-d2a3eb03-5b1c-4286-9466-5a82c432a7e2'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:53:41.062309 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-d2a3eb03-5b1c-4286-9466-5a82c432a7e2 admin admin] The token cache is being invalidate because project 25139a07847c44f89448109ec8fadd56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:53:41.062634 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-d2a3eb03-5b1c-4286-9466-5a82c432a7e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:53:41.063293 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-d2a3eb03-5b1c-4286-9466-5a82c432a7e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:53:41.065058 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1246/2495] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:53:40 2022] DELETE /identity/v3/projects/25139a07847c44f89448109ec8fadd56 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:53:50.690722 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=4 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:53:51.059338 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:54:02.276284 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca720ac-77e2-4d7a-be88-25ca8f672667 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:54:02.277707 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-7ca720ac-77e2-4d7a-be88-25ca8f672667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:54:02.334830 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca720ac-77e2-4d7a-be88-25ca8f672667 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:54:02.382458 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ebef08-2c7d-47bd-bd88-6c5eaa1f6569 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:54:02.383063 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebef08-2c7d-47bd-bd88-6c5eaa1f6569 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:54:02.383344 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebef08-2c7d-47bd-bd88-6c5eaa1f6569 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:54:02.383616 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebef08-2c7d-47bd-bd88-6c5eaa1f6569 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:54:02.385380 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-99ebef08-2c7d-47bd-bd88-6c5eaa1f6569 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:54:02.446645 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ebef08-2c7d-47bd-bd88-6c5eaa1f6569 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:54:02.448153 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ebef08-2c7d-47bd-bd88-6c5eaa1f6569 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:54:02.458715 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1250/2496] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:54:02 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:54:05.954826 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3eb03-5b1c-4286-9466-5a82c432a7e2 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:54:05.956321 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-d2a3eb03-5b1c-4286-9466-5a82c432a7e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:54:05.968444 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3eb03-5b1c-4286-9466-5a82c432a7e2 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:54:05.972610 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8d0fc-fee4-4afa-8f13-87e1ae4c8fd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:54:05.973219 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8d0fc-fee4-4afa-8f13-87e1ae4c8fd0 service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:54:05.973480 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8d0fc-fee4-4afa-8f13-87e1ae4c8fd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:54:05.973752 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8d0fc-fee4-4afa-8f13-87e1ae4c8fd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:54:05.975518 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-c8a8d0fc-fee4-4afa-8f13-87e1ae4c8fd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:54:05.984894 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a8d0fc-fee4-4afa-8f13-87e1ae4c8fd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:54:05.986976 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a8d0fc-fee4-4afa-8f13-87e1ae4c8fd0 service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:54:05.994754 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1247/2497] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:54:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:54:06.947946 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f1ca2-4bfa-4575-8c89-989e9345520a None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:54:06.948470 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f1ca2-4bfa-4575-8c89-989e9345520a None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:54:06.948647 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f1ca2-4bfa-4575-8c89-989e9345520a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:54:06.967834 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-355f1ca2-4bfa-4575-8c89-989e9345520a None None] MFA Rules not processed for user `64fdf46f185c48558ad1a637898745a9`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:54:06.970176 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-355f1ca2-4bfa-4575-8c89-989e9345520a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:54:07.024533 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1251/2498] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:54:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:54:15.991228 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=6 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:54:17.017170 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=95 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:54:47.408815 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8d0fc-fee4-4afa-8f13-87e1ae4c8fd0 service neutron] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:54:47.410581 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-c8a8d0fc-fee4-4afa-8f13-87e1ae4c8fd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:54:47.516822 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8d0fc-fee4-4afa-8f13-87e1ae4c8fd0 service neutron] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:54:47.523335 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f6203-03c4-4e16-a2a2-6f7aa45e9613 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:54:47.525011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f6203-03c4-4e16-a2a2-6f7aa45e9613 service nova] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:54:47.525525 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f6203-03c4-4e16-a2a2-6f7aa45e9613 service nova] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:54:47.526128 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f6203-03c4-4e16-a2a2-6f7aa45e9613 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:54:47.528408 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-041f6203-03c4-4e16-a2a2-6f7aa45e9613 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:54:47.544331 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041f6203-03c4-4e16-a2a2-6f7aa45e9613 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:54:47.546510 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041f6203-03c4-4e16-a2a2-6f7aa45e9613 service nova] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:54:47.563627 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1248/2499] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:54:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:54:52.289366 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f1ca2-4bfa-4575-8c89-989e9345520a None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:54:52.299292 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f1ca2-4bfa-4575-8c89-989e9345520a None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:54:52.302820 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba39d84a-9cae-44b9-bf6f-b32dd528fc37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:54:52.303416 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39d84a-9cae-44b9-bf6f-b32dd528fc37 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:54:52.303693 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39d84a-9cae-44b9-bf6f-b32dd528fc37 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:54:52.304009 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39d84a-9cae-44b9-bf6f-b32dd528fc37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:54:52.305702 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-ba39d84a-9cae-44b9-bf6f-b32dd528fc37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:54:52.368612 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba39d84a-9cae-44b9-bf6f-b32dd528fc37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:54:52.369979 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba39d84a-9cae-44b9-bf6f-b32dd528fc37 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:54:52.379525 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1252/2500] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:54:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:54:52.624345 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f6203-03c4-4e16-a2a2-6f7aa45e9613 service nova] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:54:52.635837 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f6203-03c4-4e16-a2a2-6f7aa45e9613 service nova] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:54:52.641310 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496bf4c0-4e9f-4e60-a896-440a59ce1b9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:54:52.641902 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bf4c0-4e9f-4e60-a896-440a59ce1b9c service nova] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:54:52.642179 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bf4c0-4e9f-4e60-a896-440a59ce1b9c service nova] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:54:52.642452 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bf4c0-4e9f-4e60-a896-440a59ce1b9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:54:52.644799 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-496bf4c0-4e9f-4e60-a896-440a59ce1b9c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:54:52.725927 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496bf4c0-4e9f-4e60-a896-440a59ce1b9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:54:52.727530 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496bf4c0-4e9f-4e60-a896-440a59ce1b9c service nova] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:54:52.745826 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1249/2501] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:54:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:54:56.289681 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba39d84a-9cae-44b9-bf6f-b32dd528fc37 service neutron] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:54:56.291397 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-ba39d84a-9cae-44b9-bf6f-b32dd528fc37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:54:56.303800 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba39d84a-9cae-44b9-bf6f-b32dd528fc37 service neutron] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:54:56.308481 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff1515c-28ea-4fc0-b5fc-b599edeba896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:54:56.309211 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff1515c-28ea-4fc0-b5fc-b599edeba896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:54:56.309489 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff1515c-28ea-4fc0-b5fc-b599edeba896 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:54:56.309759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff1515c-28ea-4fc0-b5fc-b599edeba896 admin admin] PATH_INFO: `/v3/users/047293a64a9f4857be1621e77094d367` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:54:56.310884 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff1515c-28ea-4fc0-b5fc-b599edeba896 admin admin] RBAC: Authorizing `identity:delete_user(user_id=047293a64a9f4857be1621e77094d367)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:54:56.312868 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff1515c-28ea-4fc0-b5fc-b599edeba896 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:54:56.363633 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-1ff1515c-28ea-4fc0-b5fc-b599edeba896 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '047293a64a9f4857be1621e77094d367'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:54:56.364296 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-1ff1515c-28ea-4fc0-b5fc-b599edeba896 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '047293a64a9f4857be1621e77094d367'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:54:56.376109 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-1ff1515c-28ea-4fc0-b5fc-b599edeba896 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '047293a64a9f4857be1621e77094d367'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:54:56.383663 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-1ff1515c-28ea-4fc0-b5fc-b599edeba896 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '047293a64a9f4857be1621e77094d367'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:54:56.384301 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-1ff1515c-28ea-4fc0-b5fc-b599edeba896 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '047293a64a9f4857be1621e77094d367'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:54:56.397165 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1253/2502] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:54:56 2022] DELETE /identity/v3/users/047293a64a9f4857be1621e77094d367 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:54:56.566581 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496bf4c0-4e9f-4e60-a896-440a59ce1b9c service nova] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:54:56.588364 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496bf4c0-4e9f-4e60-a896-440a59ce1b9c service nova] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:54:56.646625 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:54:56.653961 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df84609-7b39-4264-84eb-6d9affff65fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:54:56.654584 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df84609-7b39-4264-84eb-6d9affff65fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:54:56.654869 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df84609-7b39-4264-84eb-6d9affff65fc admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:54:56.655132 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df84609-7b39-4264-84eb-6d9affff65fc admin admin] PATH_INFO: `/v3/projects/ec8246e3b9904e7a8fe4a04269ab0afe` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:54:56.656287 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df84609-7b39-4264-84eb-6d9affff65fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec8246e3b9904e7a8fe4a04269ab0afe)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:54:56.658424 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df84609-7b39-4264-84eb-6d9affff65fc admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:54:56.703459 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-3df84609-7b39-4264-84eb-6d9affff65fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec8246e3b9904e7a8fe4a04269ab0afe', 'request_id': 'req-3df84609-7b39-4264-84eb-6d9affff65fc'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:54:56.708754 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-3df84609-7b39-4264-84eb-6d9affff65fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec8246e3b9904e7a8fe4a04269ab0afe', 'request_id': 'req-3df84609-7b39-4264-84eb-6d9affff65fc'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:54:56.715846 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-3df84609-7b39-4264-84eb-6d9affff65fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec8246e3b9904e7a8fe4a04269ab0afe', 'request_id': 'req-3df84609-7b39-4264-84eb-6d9affff65fc'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:54:56.726587 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-3df84609-7b39-4264-84eb-6d9affff65fc admin admin] The token cache is being invalidate because project ec8246e3b9904e7a8fe4a04269ab0afe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:54:56.726874 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-3df84609-7b39-4264-84eb-6d9affff65fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:54:56.727496 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-3df84609-7b39-4264-84eb-6d9affff65fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:54:56.728998 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1250/2503] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:54:56 2022] DELETE /identity/v3/projects/ec8246e3b9904e7a8fe4a04269ab0afe => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:54:56.733911 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa968f85-17f4-4717-9c25-a529f6d31bea None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:54:56.734194 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa968f85-17f4-4717-9c25-a529f6d31bea None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:54:56.734464 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa968f85-17f4-4717-9c25-a529f6d31bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:54:56.754764 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-aa968f85-17f4-4717-9c25-a529f6d31bea None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:54:56.755944 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-aa968f85-17f4-4717-9c25-a529f6d31bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:54:56.812321 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1254/2504] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:54:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:54:56.820770 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a6fb2-21fd-472f-a967-941f3a1ce5c4 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:54:56.821056 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a6fb2-21fd-472f-a967-941f3a1ce5c4 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:54:56.821354 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a6fb2-21fd-472f-a967-941f3a1ce5c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:54:56.840297 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-207a6fb2-21fd-472f-a967-941f3a1ce5c4 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:54:56.841499 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-207a6fb2-21fd-472f-a967-941f3a1ce5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:54:56.849158 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1251/2505] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:54:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:54:56.853173 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa968f85-17f4-4717-9c25-a529f6d31bea None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:54:56.863377 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa968f85-17f4-4717-9c25-a529f6d31bea None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:54:56.869299 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a5676-2527-4d93-be5c-5f22d4b7d1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:54:56.869897 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a5676-2527-4d93-be5c-5f22d4b7d1e2 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:54:56.870172 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a5676-2527-4d93-be5c-5f22d4b7d1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:54:56.870438 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a5676-2527-4d93-be5c-5f22d4b7d1e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:54:56.871100 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a5676-2527-4d93-be5c-5f22d4b7d1e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:54:56.871383 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a5676-2527-4d93-be5c-5f22d4b7d1e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:54:56.873014 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a5676-2527-4d93-be5c-5f22d4b7d1e2 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:54:56.874574 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:54:56.887674 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1255/2506] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:54:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:54:56.891937 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207a6fb2-21fd-472f-a967-941f3a1ce5c4 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:54:56.901124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207a6fb2-21fd-472f-a967-941f3a1ce5c4 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:54:56.905815 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f1f08-a1ad-42cb-98f1-a52639a9f366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:54:56.906414 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f1f08-a1ad-42cb-98f1-a52639a9f366 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:54:56.906695 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f1f08-a1ad-42cb-98f1-a52639a9f366 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:54:56.906970 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f1f08-a1ad-42cb-98f1-a52639a9f366 admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:54:56.907603 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6f1f08-a1ad-42cb-98f1-a52639a9f366 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:54:56.911017 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6f1f08-a1ad-42cb-98f1-a52639a9f366 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:54:56.948337 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1252/2507] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:54:56 2022] POST /identity/v3/projects => generated 365 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:54:56.952581 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a5676-2527-4d93-be5c-5f22d4b7d1e2 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:54:56.962855 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a5676-2527-4d93-be5c-5f22d4b7d1e2 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:54:57.018725 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49aeebf-fdef-40fc-adad-22a691d5642e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:54:57.019331 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49aeebf-fdef-40fc-adad-22a691d5642e admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:54:57.019741 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49aeebf-fdef-40fc-adad-22a691d5642e admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:54:57.019927 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49aeebf-fdef-40fc-adad-22a691d5642e admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:54:57.024571 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49aeebf-fdef-40fc-adad-22a691d5642e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:54:57.026477 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49aeebf-fdef-40fc-adad-22a691d5642e admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:54:57.050973 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1256/2508] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:54:56 2022] POST /identity/v3/users => generated 283 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:54:57.055014 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f1f08-a1ad-42cb-98f1-a52639a9f366 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:54:57.065719 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f1f08-a1ad-42cb-98f1-a52639a9f366 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:54:57.070742 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d901d-1dce-4b10-b9da-498ecc286e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:54:57.071450 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d901d-1dce-4b10-b9da-498ecc286e2f admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:54:57.071791 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d901d-1dce-4b10-b9da-498ecc286e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:54:57.072221 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d901d-1dce-4b10-b9da-498ecc286e2f admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:54:57.072941 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65d901d-1dce-4b10-b9da-498ecc286e2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:54:57.074608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65d901d-1dce-4b10-b9da-498ecc286e2f admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:54:57.085059 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1253/2509] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:54:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:54:57.089210 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49aeebf-fdef-40fc-adad-22a691d5642e admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:54:57.100820 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49aeebf-fdef-40fc-adad-22a691d5642e admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:54:57.105790 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e955aa-ebfa-4e2c-be26-2ba0c6c982c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:54:57.106537 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e955aa-ebfa-4e2c-be26-2ba0c6c982c6 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:54:57.106822 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e955aa-ebfa-4e2c-be26-2ba0c6c982c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:54:57.107115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e955aa-ebfa-4e2c-be26-2ba0c6c982c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:54:57.107859 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e955aa-ebfa-4e2c-be26-2ba0c6c982c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:54:57.109667 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e955aa-ebfa-4e2c-be26-2ba0c6c982c6 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:54:57.121394 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1257/2510] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:54:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:54:57.125551 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d901d-1dce-4b10-b9da-498ecc286e2f admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:54:57.136782 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d901d-1dce-4b10-b9da-498ecc286e2f admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:54:57.141371 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd16b0-b9de-40bb-9867-1f057f631a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:54:57.142012 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd16b0-b9de-40bb-9867-1f057f631a45 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:54:57.142290 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd16b0-b9de-40bb-9867-1f057f631a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:54:57.142562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd16b0-b9de-40bb-9867-1f057f631a45 admin admin] PATH_INFO: `/v3/projects/5892d4415180410c91c06096c317e4a0/users/d5024ed7e9a34f58a630f608bdd8691f/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:54:57.157924 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcd16b0-b9de-40bb-9867-1f057f631a45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5892d4415180410c91c06096c317e4a0, user_id=d5024ed7e9a34f58a630f608bdd8691f, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:54:57.161192 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcd16b0-b9de-40bb-9867-1f057f631a45 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:54:57.174825 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1254/2511] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:54:57 2022] PUT /identity/v3/projects/5892d4415180410c91c06096c317e4a0/users/d5024ed7e9a34f58a630f608bdd8691f/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:54:57.185330 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c59808-bf66-4bdf-bd61-ba6ffebf081c None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:54:57.185617 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c59808-bf66-4bdf-bd61-ba6ffebf081c None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:54:57.185890 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c59808-bf66-4bdf-bd61-ba6ffebf081c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:54:57.212256 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-87c59808-bf66-4bdf-bd61-ba6ffebf081c None None] MFA Rules not processed for user `d5024ed7e9a34f58a630f608bdd8691f`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:54:57.213516 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-87c59808-bf66-4bdf-bd61-ba6ffebf081c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:54:57.277619 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1258/2512] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:54:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:54:57.282173 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd16b0-b9de-40bb-9867-1f057f631a45 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:54:57.346666 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd16b0-b9de-40bb-9867-1f057f631a45 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:54:57.351899 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1dffef-8ee8-444f-a92e-acf2d543f704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:54:57.352533 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1dffef-8ee8-444f-a92e-acf2d543f704 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:54:57.352818 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1dffef-8ee8-444f-a92e-acf2d543f704 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:54:57.353202 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1dffef-8ee8-444f-a92e-acf2d543f704 admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:54:57.353746 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1dffef-8ee8-444f-a92e-acf2d543f704 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:54:57.355799 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1dffef-8ee8-444f-a92e-acf2d543f704 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:54:57.379501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:54:57.389499 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1255/2513] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:54:57 2022] POST /identity/v3/projects => generated 365 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:54:57.394512 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c59808-bf66-4bdf-bd61-ba6ffebf081c None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:54:57.405993 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c59808-bf66-4bdf-bd61-ba6ffebf081c None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:54:57.469318 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17da375b-ed3e-4d22-b55b-0a2251f36ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:54:57.469925 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da375b-ed3e-4d22-b55b-0a2251f36ed5 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:54:57.470209 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da375b-ed3e-4d22-b55b-0a2251f36ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:54:57.470477 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da375b-ed3e-4d22-b55b-0a2251f36ed5 admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:54:57.471146 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17da375b-ed3e-4d22-b55b-0a2251f36ed5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:54:57.473602 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17da375b-ed3e-4d22-b55b-0a2251f36ed5 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:54:57.482205 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:54:57.494637 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1259/2514] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:54:57 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:54:57.498596 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1dffef-8ee8-444f-a92e-acf2d543f704 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:54:57.508873 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1dffef-8ee8-444f-a92e-acf2d543f704 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:54:57.513424 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba2273-2ce6-4468-8a2f-85397cea8037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:54:57.514018 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba2273-2ce6-4468-8a2f-85397cea8037 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:54:57.514301 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba2273-2ce6-4468-8a2f-85397cea8037 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:54:57.514576 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba2273-2ce6-4468-8a2f-85397cea8037 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:54:57.515223 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba2273-2ce6-4468-8a2f-85397cea8037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:54:57.516837 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba2273-2ce6-4468-8a2f-85397cea8037 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:54:57.527808 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1256/2515] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:54:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:54:57.531882 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17da375b-ed3e-4d22-b55b-0a2251f36ed5 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:54:57.542674 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17da375b-ed3e-4d22-b55b-0a2251f36ed5 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:54:57.547180 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60421520-4056-4469-b8f0-8b9b9ccde4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:54:57.547940 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60421520-4056-4469-b8f0-8b9b9ccde4c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:54:57.548247 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60421520-4056-4469-b8f0-8b9b9ccde4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:54:57.548519 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60421520-4056-4469-b8f0-8b9b9ccde4c5 admin admin] PATH_INFO: `/v3/projects/79a6633bca3e4e9dba8446b9dfb1e667/users/547f0f5cfaf7423593a77dcccc703472/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:54:57.563002 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60421520-4056-4469-b8f0-8b9b9ccde4c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79a6633bca3e4e9dba8446b9dfb1e667, user_id=547f0f5cfaf7423593a77dcccc703472, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:54:57.565731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60421520-4056-4469-b8f0-8b9b9ccde4c5 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:54:57.578596 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1260/2516] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:54:57 2022] PUT /identity/v3/projects/79a6633bca3e4e9dba8446b9dfb1e667/users/547f0f5cfaf7423593a77dcccc703472/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:54:57.583146 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba2273-2ce6-4468-8a2f-85397cea8037 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:54:57.594036 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba2273-2ce6-4468-8a2f-85397cea8037 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:54:57.649797 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c120f-219f-4436-93ce-c8f55619ec73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:54:57.650411 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c120f-219f-4436-93ce-c8f55619ec73 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:54:57.650688 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c120f-219f-4436-93ce-c8f55619ec73 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:54:57.650984 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c120f-219f-4436-93ce-c8f55619ec73 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:54:57.651629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1c120f-219f-4436-93ce-c8f55619ec73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:54:57.653247 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1c120f-219f-4436-93ce-c8f55619ec73 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:54:57.664416 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1257/2517] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:54:57 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:54:57.669145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60421520-4056-4469-b8f0-8b9b9ccde4c5 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:54:57.679782 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60421520-4056-4469-b8f0-8b9b9ccde4c5 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:54:57.684498 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36617173-fe03-49f6-9a75-cb7d50cd8e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:54:57.685144 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36617173-fe03-49f6-9a75-cb7d50cd8e3d admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:54:57.685427 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36617173-fe03-49f6-9a75-cb7d50cd8e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:54:57.685706 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36617173-fe03-49f6-9a75-cb7d50cd8e3d admin admin] PATH_INFO: `/v3/projects/79a6633bca3e4e9dba8446b9dfb1e667/users/547f0f5cfaf7423593a77dcccc703472/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:54:57.687632 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36617173-fe03-49f6-9a75-cb7d50cd8e3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=79a6633bca3e4e9dba8446b9dfb1e667, user_id=547f0f5cfaf7423593a77dcccc703472, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:54:57.690830 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36617173-fe03-49f6-9a75-cb7d50cd8e3d admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:54:57.700943 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1261/2518] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:54:57 2022] PUT /identity/v3/projects/79a6633bca3e4e9dba8446b9dfb1e667/users/547f0f5cfaf7423593a77dcccc703472/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:54:57.712870 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d146b57-35de-47e7-80a9-415a8dfe1206 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:54:57.713156 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d146b57-35de-47e7-80a9-415a8dfe1206 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:54:57.713429 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d146b57-35de-47e7-80a9-415a8dfe1206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:54:57.740007 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-3d146b57-35de-47e7-80a9-415a8dfe1206 None None] MFA Rules not processed for user `547f0f5cfaf7423593a77dcccc703472`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:54:57.741168 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-3d146b57-35de-47e7-80a9-415a8dfe1206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:54:57.807943 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1258/2519] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:54:57 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:54:57.819691 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36617173-fe03-49f6-9a75-cb7d50cd8e3d admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:54:57.821523 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-36617173-fe03-49f6-9a75-cb7d50cd8e3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:54:57.878991 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36617173-fe03-49f6-9a75-cb7d50cd8e3d admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:54:57.882795 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d492908-f989-4910-bd2b-3d67c7b960b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:54:57.883385 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d492908-f989-4910-bd2b-3d67c7b960b4 service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:54:57.883659 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d492908-f989-4910-bd2b-3d67c7b960b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:54:57.883924 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d492908-f989-4910-bd2b-3d67c7b960b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:54:57.892638 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d492908-f989-4910-bd2b-3d67c7b960b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:54:57.894675 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d492908-f989-4910-bd2b-3d67c7b960b4 service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:54:57.903775 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1262/2520] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:54:57 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:54:58.304242 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d146b57-35de-47e7-80a9-415a8dfe1206 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:54:58.365647 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d146b57-35de-47e7-80a9-415a8dfe1206 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:54:58.370113 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93dde8-f4a7-4297-aa15-2476bd55f1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:54:58.370733 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93dde8-f4a7-4297-aa15-2476bd55f1c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:54:58.371012 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93dde8-f4a7-4297-aa15-2476bd55f1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:54:58.371310 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93dde8-f4a7-4297-aa15-2476bd55f1c2 admin admin] PATH_INFO: `/v3/users/d5024ed7e9a34f58a630f608bdd8691f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:54:58.372441 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93dde8-f4a7-4297-aa15-2476bd55f1c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5024ed7e9a34f58a630f608bdd8691f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:54:58.374360 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93dde8-f4a7-4297-aa15-2476bd55f1c2 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:54:58.385163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:54:58.421403 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-9b93dde8-f4a7-4297-aa15-2476bd55f1c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5024ed7e9a34f58a630f608bdd8691f'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:54:58.422042 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-9b93dde8-f4a7-4297-aa15-2476bd55f1c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5024ed7e9a34f58a630f608bdd8691f'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:54:58.422638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-9b93dde8-f4a7-4297-aa15-2476bd55f1c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5024ed7e9a34f58a630f608bdd8691f'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:54:58.430165 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-9b93dde8-f4a7-4297-aa15-2476bd55f1c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5024ed7e9a34f58a630f608bdd8691f'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:54:58.437076 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-9b93dde8-f4a7-4297-aa15-2476bd55f1c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5024ed7e9a34f58a630f608bdd8691f'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:54:58.466025 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1259/2521] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:54:58 2022] DELETE /identity/v3/users/d5024ed7e9a34f58a630f608bdd8691f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:54:58.470380 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1aa2f-bff5-49ca-ac58-e48e3d50b8a9 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:54:58.470859 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1aa2f-bff5-49ca-ac58-e48e3d50b8a9 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:54:58.470962 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1aa2f-bff5-49ca-ac58-e48e3d50b8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:54:58.491119 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-aaf1aa2f-bff5-49ca-ac58-e48e3d50b8a9 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:54:58.492331 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-aaf1aa2f-bff5-49ca-ac58-e48e3d50b8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:54:58.549956 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1263/2522] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:54:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:54:58.559723 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93dde8-f4a7-4297-aa15-2476bd55f1c2 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:54:58.561153 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-9b93dde8-f4a7-4297-aa15-2476bd55f1c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:54:58.617940 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93dde8-f4a7-4297-aa15-2476bd55f1c2 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:54:58.621531 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ed2fc-1fbb-43ef-afdd-8c1b793ef78b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:54:58.622123 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ed2fc-1fbb-43ef-afdd-8c1b793ef78b service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:54:58.622399 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ed2fc-1fbb-43ef-afdd-8c1b793ef78b service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:54:58.622668 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ed2fc-1fbb-43ef-afdd-8c1b793ef78b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:54:58.631463 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ed2fc-1fbb-43ef-afdd-8c1b793ef78b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:54:58.632919 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ed2fc-1fbb-43ef-afdd-8c1b793ef78b service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:54:58.642184 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1260/2523] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:54:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:54:58.913305 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1aa2f-bff5-49ca-ac58-e48e3d50b8a9 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:54:58.914966 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-aaf1aa2f-bff5-49ca-ac58-e48e3d50b8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:54:58.931480 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1aa2f-bff5-49ca-ac58-e48e3d50b8a9 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:54:58.936429 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa76906-0612-4b35-8754-f44a1206b457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:54:58.937138 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa76906-0612-4b35-8754-f44a1206b457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:54:58.937357 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa76906-0612-4b35-8754-f44a1206b457 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:54:58.937673 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa76906-0612-4b35-8754-f44a1206b457 admin admin] PATH_INFO: `/v3/projects/5892d4415180410c91c06096c317e4a0` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:54:58.939331 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa76906-0612-4b35-8754-f44a1206b457 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5892d4415180410c91c06096c317e4a0)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:54:58.942127 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa76906-0612-4b35-8754-f44a1206b457 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:54:59.001192 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-6aa76906-0612-4b35-8754-f44a1206b457 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5892d4415180410c91c06096c317e4a0', 'request_id': 'req-6aa76906-0612-4b35-8754-f44a1206b457'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:54:59.003858 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:INSERT=4,keystone:SELECT=91,keystone:DELETE=5 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:54:59.008825 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-6aa76906-0612-4b35-8754-f44a1206b457 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5892d4415180410c91c06096c317e4a0', 'request_id': 'req-6aa76906-0612-4b35-8754-f44a1206b457'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:54:59.018176 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-6aa76906-0612-4b35-8754-f44a1206b457 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5892d4415180410c91c06096c317e4a0', 'request_id': 'req-6aa76906-0612-4b35-8754-f44a1206b457'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:54:59.029975 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-6aa76906-0612-4b35-8754-f44a1206b457 admin admin] The token cache is being invalidate because project 5892d4415180410c91c06096c317e4a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:54:59.030262 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-6aa76906-0612-4b35-8754-f44a1206b457 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:54:59.030899 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-6aa76906-0612-4b35-8754-f44a1206b457 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:54:59.032580 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1264/2524] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:54:58 2022] DELETE /identity/v3/projects/5892d4415180410c91c06096c317e4a0 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:54:59.036814 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ed2fc-1fbb-43ef-afdd-8c1b793ef78b service neutron] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:54:59.047014 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ed2fc-1fbb-43ef-afdd-8c1b793ef78b service neutron] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:54:59.103158 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83368c3a-0e14-45f4-b798-907689dd3534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:54:59.103789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83368c3a-0e14-45f4-b798-907689dd3534 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:54:59.104105 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83368c3a-0e14-45f4-b798-907689dd3534 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:54:59.104385 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83368c3a-0e14-45f4-b798-907689dd3534 admin admin] PATH_INFO: `/v3/users/547f0f5cfaf7423593a77dcccc703472` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:54:59.105503 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83368c3a-0e14-45f4-b798-907689dd3534 admin admin] RBAC: Authorizing `identity:delete_user(user_id=547f0f5cfaf7423593a77dcccc703472)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:54:59.107442 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83368c3a-0e14-45f4-b798-907689dd3534 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:54:59.141169 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:54:59.154389 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-83368c3a-0e14-45f4-b798-907689dd3534 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '547f0f5cfaf7423593a77dcccc703472'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:54:59.155056 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-83368c3a-0e14-45f4-b798-907689dd3534 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '547f0f5cfaf7423593a77dcccc703472'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:54:59.155702 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-83368c3a-0e14-45f4-b798-907689dd3534 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '547f0f5cfaf7423593a77dcccc703472'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:54:59.163227 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-83368c3a-0e14-45f4-b798-907689dd3534 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '547f0f5cfaf7423593a77dcccc703472'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:54:59.170419 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-83368c3a-0e14-45f4-b798-907689dd3534 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '547f0f5cfaf7423593a77dcccc703472'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:54:59.187752 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1261/2525] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:54:59 2022] DELETE /identity/v3/users/547f0f5cfaf7423593a77dcccc703472 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:54:59.464065 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa76906-0612-4b35-8754-f44a1206b457 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:54:59.465501 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-6aa76906-0612-4b35-8754-f44a1206b457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:54:59.480623 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa76906-0612-4b35-8754-f44a1206b457 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:54:59.543714 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3bcc8b-29ed-46a7-be46-75d4f5467d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:54:59.544382 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bcc8b-29ed-46a7-be46-75d4f5467d1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:54:59.544830 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bcc8b-29ed-46a7-be46-75d4f5467d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:54:59.545107 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bcc8b-29ed-46a7-be46-75d4f5467d1d admin admin] PATH_INFO: `/v3/projects/79a6633bca3e4e9dba8446b9dfb1e667` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:54:59.546232 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3bcc8b-29ed-46a7-be46-75d4f5467d1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=79a6633bca3e4e9dba8446b9dfb1e667)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:54:59.548240 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3bcc8b-29ed-46a7-be46-75d4f5467d1d admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:54:59.594327 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-3c3bcc8b-29ed-46a7-be46-75d4f5467d1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79a6633bca3e4e9dba8446b9dfb1e667', 'request_id': 'req-3c3bcc8b-29ed-46a7-be46-75d4f5467d1d'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:54:59.599703 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-3c3bcc8b-29ed-46a7-be46-75d4f5467d1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79a6633bca3e4e9dba8446b9dfb1e667', 'request_id': 'req-3c3bcc8b-29ed-46a7-be46-75d4f5467d1d'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:54:59.606943 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-3c3bcc8b-29ed-46a7-be46-75d4f5467d1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79a6633bca3e4e9dba8446b9dfb1e667', 'request_id': 'req-3c3bcc8b-29ed-46a7-be46-75d4f5467d1d'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:54:59.618601 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-3c3bcc8b-29ed-46a7-be46-75d4f5467d1d admin admin] The token cache is being invalidate because project 79a6633bca3e4e9dba8446b9dfb1e667 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:54:59.618897 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-3c3bcc8b-29ed-46a7-be46-75d4f5467d1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:54:59.619522 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-3c3bcc8b-29ed-46a7-be46-75d4f5467d1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:54:59.621035 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1265/2526] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:54:59 2022] DELETE /identity/v3/projects/79a6633bca3e4e9dba8446b9dfb1e667 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:54:59.626105 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9a247-bc47-4f28-ba68-a8ac0f075083 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:54:59.626556 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9a247-bc47-4f28-ba68-a8ac0f075083 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:54:59.626672 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9a247-bc47-4f28-ba68-a8ac0f075083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:54:59.646391 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-a5d9a247-bc47-4f28-ba68-a8ac0f075083 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:54:59.647585 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-a5d9a247-bc47-4f28-ba68-a8ac0f075083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:54:59.706942 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1262/2527] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:54:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:54:59.714777 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87068090-e454-4d21-b321-343c355de796 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:54:59.715062 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87068090-e454-4d21-b321-343c355de796 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:54:59.715331 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87068090-e454-4d21-b321-343c355de796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:54:59.734362 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-87068090-e454-4d21-b321-343c355de796 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:54:59.735554 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-87068090-e454-4d21-b321-343c355de796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:54:59.744026 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1266/2528] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:54:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:54:59.748160 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d9a247-bc47-4f28-ba68-a8ac0f075083 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:54:59.758786 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d9a247-bc47-4f28-ba68-a8ac0f075083 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:54:59.763487 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b89056-c1b1-44bc-862c-ee307fd39c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:54:59.764120 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b89056-c1b1-44bc-862c-ee307fd39c82 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:54:59.764397 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b89056-c1b1-44bc-862c-ee307fd39c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:54:59.764666 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b89056-c1b1-44bc-862c-ee307fd39c82 admin admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:54:59.765315 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b89056-c1b1-44bc-862c-ee307fd39c82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:54:59.765616 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b89056-c1b1-44bc-862c-ee307fd39c82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:54:59.767214 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b89056-c1b1-44bc-862c-ee307fd39c82 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:54:59.778894 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1263/2529] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:54:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:54:59.783177 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87068090-e454-4d21-b321-343c355de796 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:54:59.792760 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87068090-e454-4d21-b321-343c355de796 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:54:59.797474 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20625353-4907-4a1a-b18b-a0ad9139b8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:54:59.798137 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20625353-4907-4a1a-b18b-a0ad9139b8fe admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:54:59.798497 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20625353-4907-4a1a-b18b-a0ad9139b8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:54:59.798857 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20625353-4907-4a1a-b18b-a0ad9139b8fe admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:54:59.800078 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20625353-4907-4a1a-b18b-a0ad9139b8fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:54:59.802035 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20625353-4907-4a1a-b18b-a0ad9139b8fe admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:54:59.828958 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1267/2530] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:54:59 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:54:59.833482 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b89056-c1b1-44bc-862c-ee307fd39c82 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:54:59.843375 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b89056-c1b1-44bc-862c-ee307fd39c82 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:54:59.899352 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222438b9-f2bb-4f20-8cec-a4356622215c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:54:59.899859 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222438b9-f2bb-4f20-8cec-a4356622215c admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:54:59.900166 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222438b9-f2bb-4f20-8cec-a4356622215c admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:54:59.900439 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222438b9-f2bb-4f20-8cec-a4356622215c admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:54:59.901067 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222438b9-f2bb-4f20-8cec-a4356622215c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:54:59.902982 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222438b9-f2bb-4f20-8cec-a4356622215c admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:54:59.924208 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1264/2531] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:54:59 2022] POST /identity/v3/users => generated 284 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:54:59.928798 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20625353-4907-4a1a-b18b-a0ad9139b8fe admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:54:59.939217 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20625353-4907-4a1a-b18b-a0ad9139b8fe admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:54:59.943830 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b3c44c-9a76-4621-b21f-ed14cfde29b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:54:59.944453 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3c44c-9a76-4621-b21f-ed14cfde29b8 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:54:59.944733 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3c44c-9a76-4621-b21f-ed14cfde29b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:54:59.945004 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3c44c-9a76-4621-b21f-ed14cfde29b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:54:59.945640 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b3c44c-9a76-4621-b21f-ed14cfde29b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:54:59.947247 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b3c44c-9a76-4621-b21f-ed14cfde29b8 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:54:59.957016 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1268/2532] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:54:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:54:59.960959 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222438b9-f2bb-4f20-8cec-a4356622215c admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:54:59.971060 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222438b9-f2bb-4f20-8cec-a4356622215c admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:54:59.976284 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7283c17b-a76b-4288-bf6f-b1c3f43a02c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:54:59.976873 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283c17b-a76b-4288-bf6f-b1c3f43a02c1 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:54:59.977157 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283c17b-a76b-4288-bf6f-b1c3f43a02c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:54:59.977430 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283c17b-a76b-4288-bf6f-b1c3f43a02c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:54:59.978069 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7283c17b-a76b-4288-bf6f-b1c3f43a02c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:54:59.979650 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7283c17b-a76b-4288-bf6f-b1c3f43a02c1 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:54:59.990698 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1265/2533] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:54:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:54:59.994693 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b3c44c-9a76-4621-b21f-ed14cfde29b8 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:00.004584 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b3c44c-9a76-4621-b21f-ed14cfde29b8 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:00.009057 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe098e62-495c-4238-919d-12fdc0ad7c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:00.009686 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe098e62-495c-4238-919d-12fdc0ad7c2a admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:00.009960 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe098e62-495c-4238-919d-12fdc0ad7c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:00.010236 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe098e62-495c-4238-919d-12fdc0ad7c2a admin admin] PATH_INFO: `/v3/projects/2d8336ef8dda4a44a77d20c918353c12/users/4a923309e9a9454cb165605392eba2b5/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:00.025856 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe098e62-495c-4238-919d-12fdc0ad7c2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d8336ef8dda4a44a77d20c918353c12, user_id=4a923309e9a9454cb165605392eba2b5, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:00.028610 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe098e62-495c-4238-919d-12fdc0ad7c2a admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:00.042967 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1269/2534] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:54:59 2022] PUT /identity/v3/projects/2d8336ef8dda4a44a77d20c918353c12/users/4a923309e9a9454cb165605392eba2b5/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:00.054889 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7720c-1b4a-41de-8e45-385930191afb None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:00.055178 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7720c-1b4a-41de-8e45-385930191afb None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:00.055484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7720c-1b4a-41de-8e45-385930191afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:00.082415 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-96e7720c-1b4a-41de-8e45-385930191afb None None] MFA Rules not processed for user `4a923309e9a9454cb165605392eba2b5`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:00.083655 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-96e7720c-1b4a-41de-8e45-385930191afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:00.087551 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:00.148666 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1266/2535] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:55:00.153154 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe098e62-495c-4238-919d-12fdc0ad7c2a admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:00.162485 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:00.216368 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe098e62-495c-4238-919d-12fdc0ad7c2a admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:00.221110 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f9c2e-69d5-463b-b7f2-1ee285f158fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:00.222221 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f9c2e-69d5-463b-b7f2-1ee285f158fb admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:00.222499 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f9c2e-69d5-463b-b7f2-1ee285f158fb admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:00.222769 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f9c2e-69d5-463b-b7f2-1ee285f158fb admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:00.223394 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6f9c2e-69d5-463b-b7f2-1ee285f158fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:00.225375 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6f9c2e-69d5-463b-b7f2-1ee285f158fb admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:00.252314 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1270/2536] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:55:00 2022] POST /identity/v3/projects => generated 367 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:55:00.256334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e7720c-1b4a-41de-8e45-385930191afb None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:00.267060 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e7720c-1b4a-41de-8e45-385930191afb None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:00.322308 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb998f-d889-480e-8007-206fc39fa6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:00.322912 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb998f-d889-480e-8007-206fc39fa6b5 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:00.323193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb998f-d889-480e-8007-206fc39fa6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:00.323468 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb998f-d889-480e-8007-206fc39fa6b5 admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:00.324114 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eb998f-d889-480e-8007-206fc39fa6b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:00.326019 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eb998f-d889-480e-8007-206fc39fa6b5 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:00.347276 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1267/2537] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:55:00 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:55:00.351259 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f9c2e-69d5-463b-b7f2-1ee285f158fb admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:00.361639 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f9c2e-69d5-463b-b7f2-1ee285f158fb admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:00.366283 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a1e6d2-85f3-43a2-892c-042af8c6115e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:00.366871 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1e6d2-85f3-43a2-892c-042af8c6115e admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:00.367150 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1e6d2-85f3-43a2-892c-042af8c6115e admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:00.367418 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1e6d2-85f3-43a2-892c-042af8c6115e admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:00.368137 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a1e6d2-85f3-43a2-892c-042af8c6115e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:00.369732 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a1e6d2-85f3-43a2-892c-042af8c6115e admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:00.380071 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1271/2538] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:55:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:55:00.384053 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb998f-d889-480e-8007-206fc39fa6b5 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:00.394633 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb998f-d889-480e-8007-206fc39fa6b5 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:00.399279 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff611a-f156-45bb-8373-7d08bf6cc413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:00.399928 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff611a-f156-45bb-8373-7d08bf6cc413 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:00.400239 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff611a-f156-45bb-8373-7d08bf6cc413 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:00.400514 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff611a-f156-45bb-8373-7d08bf6cc413 admin admin] PATH_INFO: `/v3/projects/7583f523ece04f0291e0e2692be48555/users/821ef3fbb2d447cb946d5b5a6abb19b7/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:00.415420 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ff611a-f156-45bb-8373-7d08bf6cc413 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7583f523ece04f0291e0e2692be48555, user_id=821ef3fbb2d447cb946d5b5a6abb19b7, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:00.418313 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ff611a-f156-45bb-8373-7d08bf6cc413 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:00.430885 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1268/2539] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:55:00 2022] PUT /identity/v3/projects/7583f523ece04f0291e0e2692be48555/users/821ef3fbb2d447cb946d5b5a6abb19b7/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:00.434902 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a1e6d2-85f3-43a2-892c-042af8c6115e admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:00.444927 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a1e6d2-85f3-43a2-892c-042af8c6115e admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:00.502635 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de774d6-2544-4e48-bc1d-7bf23d6fbba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:00.503287 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de774d6-2544-4e48-bc1d-7bf23d6fbba6 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:00.503516 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de774d6-2544-4e48-bc1d-7bf23d6fbba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:00.503794 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de774d6-2544-4e48-bc1d-7bf23d6fbba6 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:00.504464 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de774d6-2544-4e48-bc1d-7bf23d6fbba6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:00.506243 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de774d6-2544-4e48-bc1d-7bf23d6fbba6 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:00.517197 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1272/2540] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:55:00 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:55:00.522092 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff611a-f156-45bb-8373-7d08bf6cc413 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:00.532316 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff611a-f156-45bb-8373-7d08bf6cc413 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:00.537608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2e1f9a-f366-4ecb-882a-0e0f73d73978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:00.538248 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e1f9a-f366-4ecb-882a-0e0f73d73978 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:00.538524 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e1f9a-f366-4ecb-882a-0e0f73d73978 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:00.538819 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e1f9a-f366-4ecb-882a-0e0f73d73978 admin admin] PATH_INFO: `/v3/projects/7583f523ece04f0291e0e2692be48555/users/821ef3fbb2d447cb946d5b5a6abb19b7/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:00.540752 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2e1f9a-f366-4ecb-882a-0e0f73d73978 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7583f523ece04f0291e0e2692be48555, user_id=821ef3fbb2d447cb946d5b5a6abb19b7, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:00.543628 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2e1f9a-f366-4ecb-882a-0e0f73d73978 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:00.553586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1269/2541] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:55:00 2022] PUT /identity/v3/projects/7583f523ece04f0291e0e2692be48555/users/821ef3fbb2d447cb946d5b5a6abb19b7/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:00.563758 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170035b-397d-4bab-a818-a488b46ef235 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:00.564079 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170035b-397d-4bab-a818-a488b46ef235 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:00.564359 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170035b-397d-4bab-a818-a488b46ef235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:00.591634 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-d170035b-397d-4bab-a818-a488b46ef235 None None] MFA Rules not processed for user `821ef3fbb2d447cb946d5b5a6abb19b7`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:00.592863 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-d170035b-397d-4bab-a818-a488b46ef235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:00.659502 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1273/2542] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:00 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:55:00.670286 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2e1f9a-f366-4ecb-882a-0e0f73d73978 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:00.671915 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-3e2e1f9a-f366-4ecb-882a-0e0f73d73978 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:00.727589 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2e1f9a-f366-4ecb-882a-0e0f73d73978 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:00.731207 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12039b9-47c9-4b56-a420-16e3dd211051 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:00.731803 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12039b9-47c9-4b56-a420-16e3dd211051 service cinder] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:00.732111 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12039b9-47c9-4b56-a420-16e3dd211051 service cinder] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:00.732388 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12039b9-47c9-4b56-a420-16e3dd211051 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:00.741201 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12039b9-47c9-4b56-a420-16e3dd211051 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:00.742630 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12039b9-47c9-4b56-a420-16e3dd211051 service cinder] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:00.747489 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:00.753129 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1270/2543] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:55:00 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:55:02.068606 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170035b-397d-4bab-a818-a488b46ef235 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:02.072798 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:02.133809 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170035b-397d-4bab-a818-a488b46ef235 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:02.138323 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab6cd95-63f0-4fff-8786-c69cf6419681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:02.138942 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6cd95-63f0-4fff-8786-c69cf6419681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:02.139218 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6cd95-63f0-4fff-8786-c69cf6419681 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:02.139483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6cd95-63f0-4fff-8786-c69cf6419681 admin admin] PATH_INFO: `/v3/users/4a923309e9a9454cb165605392eba2b5` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:02.140610 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab6cd95-63f0-4fff-8786-c69cf6419681 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a923309e9a9454cb165605392eba2b5)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:02.142573 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab6cd95-63f0-4fff-8786-c69cf6419681 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:02.187365 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-2ab6cd95-63f0-4fff-8786-c69cf6419681 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a923309e9a9454cb165605392eba2b5'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:02.188046 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-2ab6cd95-63f0-4fff-8786-c69cf6419681 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a923309e9a9454cb165605392eba2b5'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:02.200164 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-2ab6cd95-63f0-4fff-8786-c69cf6419681 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a923309e9a9454cb165605392eba2b5'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:02.207900 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-2ab6cd95-63f0-4fff-8786-c69cf6419681 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a923309e9a9454cb165605392eba2b5'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:02.208541 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-2ab6cd95-63f0-4fff-8786-c69cf6419681 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a923309e9a9454cb165605392eba2b5'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:02.220470 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1274/2544] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:55:02 2022] DELETE /identity/v3/users/4a923309e9a9454cb165605392eba2b5 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:02.225050 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40cb312-06a9-4a88-8590-9d1dc049bea9 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:02.225562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40cb312-06a9-4a88-8590-9d1dc049bea9 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:02.225688 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40cb312-06a9-4a88-8590-9d1dc049bea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:02.251326 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-a40cb312-06a9-4a88-8590-9d1dc049bea9 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:02.253081 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-a40cb312-06a9-4a88-8590-9d1dc049bea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:02.309314 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1271/2545] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:55:02.319663 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab6cd95-63f0-4fff-8786-c69cf6419681 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:02.321122 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-2ab6cd95-63f0-4fff-8786-c69cf6419681 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:02.381348 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab6cd95-63f0-4fff-8786-c69cf6419681 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:02.385024 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b2a2ce-72bd-48ef-a99a-f69eef018452 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:02.385629 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b2a2ce-72bd-48ef-a99a-f69eef018452 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:02.385912 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b2a2ce-72bd-48ef-a99a-f69eef018452 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:02.386183 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b2a2ce-72bd-48ef-a99a-f69eef018452 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:02.395589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b2a2ce-72bd-48ef-a99a-f69eef018452 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:02.397057 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b2a2ce-72bd-48ef-a99a-f69eef018452 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:02.406329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1275/2546] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:55:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:55:02.694298 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40cb312-06a9-4a88-8590-9d1dc049bea9 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:02.695984 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-a40cb312-06a9-4a88-8590-9d1dc049bea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:02.709229 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40cb312-06a9-4a88-8590-9d1dc049bea9 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:02.714145 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710b62d-24c3-4e7d-b310-13fa7b14b491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:02.715198 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710b62d-24c3-4e7d-b310-13fa7b14b491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:02.715808 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710b62d-24c3-4e7d-b310-13fa7b14b491 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:02.716342 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710b62d-24c3-4e7d-b310-13fa7b14b491 admin admin] PATH_INFO: `/v3/projects/2d8336ef8dda4a44a77d20c918353c12` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:02.717691 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c710b62d-24c3-4e7d-b310-13fa7b14b491 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d8336ef8dda4a44a77d20c918353c12)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:02.719883 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c710b62d-24c3-4e7d-b310-13fa7b14b491 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:02.771367 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-c710b62d-24c3-4e7d-b310-13fa7b14b491 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d8336ef8dda4a44a77d20c918353c12', 'request_id': 'req-c710b62d-24c3-4e7d-b310-13fa7b14b491'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:02.777555 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-c710b62d-24c3-4e7d-b310-13fa7b14b491 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d8336ef8dda4a44a77d20c918353c12', 'request_id': 'req-c710b62d-24c3-4e7d-b310-13fa7b14b491'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:02.784943 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-c710b62d-24c3-4e7d-b310-13fa7b14b491 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d8336ef8dda4a44a77d20c918353c12', 'request_id': 'req-c710b62d-24c3-4e7d-b310-13fa7b14b491'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:02.796764 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-c710b62d-24c3-4e7d-b310-13fa7b14b491 admin admin] The token cache is being invalidate because project 2d8336ef8dda4a44a77d20c918353c12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:55:02.796953 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-c710b62d-24c3-4e7d-b310-13fa7b14b491 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:02.797588 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-c710b62d-24c3-4e7d-b310-13fa7b14b491 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:02.799890 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1272/2547] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:55:02 2022] DELETE /identity/v3/projects/2d8336ef8dda4a44a77d20c918353c12 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:02.803712 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b2a2ce-72bd-48ef-a99a-f69eef018452 service neutron] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:02.813450 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b2a2ce-72bd-48ef-a99a-f69eef018452 service neutron] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:02.834901 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:02.883457 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8046b319-61a9-4159-a971-6fb835700d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:02.884430 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046b319-61a9-4159-a971-6fb835700d84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:02.884915 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046b319-61a9-4159-a971-6fb835700d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:02.885181 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046b319-61a9-4159-a971-6fb835700d84 admin admin] PATH_INFO: `/v3/users/821ef3fbb2d447cb946d5b5a6abb19b7` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:02.886626 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8046b319-61a9-4159-a971-6fb835700d84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=821ef3fbb2d447cb946d5b5a6abb19b7)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:02.888303 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8046b319-61a9-4159-a971-6fb835700d84 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:02.945280 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-8046b319-61a9-4159-a971-6fb835700d84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '821ef3fbb2d447cb946d5b5a6abb19b7'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:02.946346 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-8046b319-61a9-4159-a971-6fb835700d84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '821ef3fbb2d447cb946d5b5a6abb19b7'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:02.959881 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-8046b319-61a9-4159-a971-6fb835700d84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '821ef3fbb2d447cb946d5b5a6abb19b7'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:02.968763 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-8046b319-61a9-4159-a971-6fb835700d84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '821ef3fbb2d447cb946d5b5a6abb19b7'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:02.969468 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-8046b319-61a9-4159-a971-6fb835700d84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '821ef3fbb2d447cb946d5b5a6abb19b7'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:02.982922 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1276/2548] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:55:02 2022] DELETE /identity/v3/users/821ef3fbb2d447cb946d5b5a6abb19b7 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:03.259825 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710b62d-24c3-4e7d-b310-13fa7b14b491 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:03.261868 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-c710b62d-24c3-4e7d-b310-13fa7b14b491 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:03.278813 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710b62d-24c3-4e7d-b310-13fa7b14b491 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:03.338826 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45daab57-0c9d-4ca3-a0f2-3cdecb165674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:03.339461 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45daab57-0c9d-4ca3-a0f2-3cdecb165674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:03.339774 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45daab57-0c9d-4ca3-a0f2-3cdecb165674 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:03.340089 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45daab57-0c9d-4ca3-a0f2-3cdecb165674 admin admin] PATH_INFO: `/v3/projects/7583f523ece04f0291e0e2692be48555` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:03.341142 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45daab57-0c9d-4ca3-a0f2-3cdecb165674 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7583f523ece04f0291e0e2692be48555)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:03.343060 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45daab57-0c9d-4ca3-a0f2-3cdecb165674 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:03.378216 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:03.398394 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-45daab57-0c9d-4ca3-a0f2-3cdecb165674 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7583f523ece04f0291e0e2692be48555', 'request_id': 'req-45daab57-0c9d-4ca3-a0f2-3cdecb165674'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:03.404134 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-45daab57-0c9d-4ca3-a0f2-3cdecb165674 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7583f523ece04f0291e0e2692be48555', 'request_id': 'req-45daab57-0c9d-4ca3-a0f2-3cdecb165674'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:03.413446 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-45daab57-0c9d-4ca3-a0f2-3cdecb165674 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7583f523ece04f0291e0e2692be48555', 'request_id': 'req-45daab57-0c9d-4ca3-a0f2-3cdecb165674'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:03.425240 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-45daab57-0c9d-4ca3-a0f2-3cdecb165674 admin admin] The token cache is being invalidate because project 7583f523ece04f0291e0e2692be48555 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:55:03.425523 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-45daab57-0c9d-4ca3-a0f2-3cdecb165674 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:03.426143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-45daab57-0c9d-4ca3-a0f2-3cdecb165674 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:03.427677 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1273/2549] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:55:03 2022] DELETE /identity/v3/projects/7583f523ece04f0291e0e2692be48555 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:03.432648 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01cdf4c-28b6-406e-beb3-be0a492f80ab None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:03.433139 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01cdf4c-28b6-406e-beb3-be0a492f80ab None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:03.433338 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01cdf4c-28b6-406e-beb3-be0a492f80ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:03.452701 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-e01cdf4c-28b6-406e-beb3-be0a492f80ab None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:03.453906 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-e01cdf4c-28b6-406e-beb3-be0a492f80ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:03.515411 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1277/2550] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:55:03.523949 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7d662-ed2d-4686-9d65-d4cdedcf2ee4 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:03.524284 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7d662-ed2d-4686-9d65-d4cdedcf2ee4 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:03.524585 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7d662-ed2d-4686-9d65-d4cdedcf2ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:03.545829 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-77d7d662-ed2d-4686-9d65-d4cdedcf2ee4 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:03.547086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-77d7d662-ed2d-4686-9d65-d4cdedcf2ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:03.555213 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1274/2551] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:55:03.559615 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01cdf4c-28b6-406e-beb3-be0a492f80ab None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:03.569753 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01cdf4c-28b6-406e-beb3-be0a492f80ab None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:03.574380 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d17026-0b55-40e0-9b46-4c34a45b59b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:03.574976 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d17026-0b55-40e0-9b46-4c34a45b59b0 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:03.575256 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d17026-0b55-40e0-9b46-4c34a45b59b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:03.575542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d17026-0b55-40e0-9b46-4c34a45b59b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:03.576209 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d17026-0b55-40e0-9b46-4c34a45b59b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:55:03.576513 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d17026-0b55-40e0-9b46-4c34a45b59b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:03.578116 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d17026-0b55-40e0-9b46-4c34a45b59b0 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:03.590648 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1278/2552] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:55:03.595012 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d7d662-ed2d-4686-9d65-d4cdedcf2ee4 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:03.605880 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d7d662-ed2d-4686-9d65-d4cdedcf2ee4 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:03.610872 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07f06a-54ff-47ea-90e0-f1f3796409eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:03.611480 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07f06a-54ff-47ea-90e0-f1f3796409eb admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:03.612092 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07f06a-54ff-47ea-90e0-f1f3796409eb admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:03.612392 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07f06a-54ff-47ea-90e0-f1f3796409eb admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:03.613034 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e07f06a-54ff-47ea-90e0-f1f3796409eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:03.613156 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d17026-0b55-40e0-9b46-4c34a45b59b0 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:03.614786 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-43d17026-0b55-40e0-9b46-4c34a45b59b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:03.614964 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e07f06a-54ff-47ea-90e0-f1f3796409eb admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:03.647769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1275/2553] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:55:03 2022] POST /identity/v3/projects => generated 387 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:55:03.649836 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:03.652008 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07f06a-54ff-47ea-90e0-f1f3796409eb admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:03.676384 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d17026-0b55-40e0-9b46-4c34a45b59b0 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:03.720468 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07f06a-54ff-47ea-90e0-f1f3796409eb admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:03.725128 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2cc38d-c180-4b91-981b-c4b48a3b7b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:03.725732 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2cc38d-c180-4b91-981b-c4b48a3b7b94 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:03.726008 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2cc38d-c180-4b91-981b-c4b48a3b7b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:03.726126 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c8c9e-1c29-4077-8f35-b012236ade69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:03.726282 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2cc38d-c180-4b91-981b-c4b48a3b7b94 admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:03.726714 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c8c9e-1c29-4077-8f35-b012236ade69 service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:03.726901 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2cc38d-c180-4b91-981b-c4b48a3b7b94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:03.727015 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c8c9e-1c29-4077-8f35-b012236ade69 service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:03.727363 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c8c9e-1c29-4077-8f35-b012236ade69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:03.728961 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2cc38d-c180-4b91-981b-c4b48a3b7b94 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:03.729195 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-207c8c9e-1c29-4077-8f35-b012236ade69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:03.753644 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1276/2554] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:55:03 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:55:03.754670 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2cc38d-c180-4b91-981b-c4b48a3b7b94 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:03.764114 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2cc38d-c180-4b91-981b-c4b48a3b7b94 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:03.769301 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37859b83-de19-432e-a95b-0685fab49e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:03.769885 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37859b83-de19-432e-a95b-0685fab49e96 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:03.770157 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37859b83-de19-432e-a95b-0685fab49e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:03.770420 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37859b83-de19-432e-a95b-0685fab49e96 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:03.771079 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37859b83-de19-432e-a95b-0685fab49e96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:03.772679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37859b83-de19-432e-a95b-0685fab49e96 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:03.782285 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1277/2555] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 23:55:03.786117 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37859b83-de19-432e-a95b-0685fab49e96 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:03.795727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37859b83-de19-432e-a95b-0685fab49e96 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:03.800501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4b570-5360-4d96-9640-acd06ed48a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:03.801083 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4b570-5360-4d96-9640-acd06ed48a63 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:03.801364 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4b570-5360-4d96-9640-acd06ed48a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:03.801643 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4b570-5360-4d96-9640-acd06ed48a63 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:03.802277 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf4b570-5360-4d96-9640-acd06ed48a63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:03.803852 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf4b570-5360-4d96-9640-acd06ed48a63 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:03.817796 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1278/2556] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 23:55:03.818018 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c8c9e-1c29-4077-8f35-b012236ade69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:03.818319 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c8c9e-1c29-4077-8f35-b012236ade69 service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:03.819177 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4b570-5360-4d96-9640-acd06ed48a63 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:03.828438 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4b570-5360-4d96-9640-acd06ed48a63 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:03.833583 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a078b3-b12b-431b-bdba-0c9b51e7b276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:03.834207 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a078b3-b12b-431b-bdba-0c9b51e7b276 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:03.834483 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a078b3-b12b-431b-bdba-0c9b51e7b276 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:03.834750 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a078b3-b12b-431b-bdba-0c9b51e7b276 admin admin] PATH_INFO: `/v3/projects/87415e7d6f32436ab995a89adac1d033/users/4756e4fb7f2d4273938edbec66226d85/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:03.837296 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1279/2557] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:55:03 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 23:55:03.851948 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a078b3-b12b-431b-bdba-0c9b51e7b276 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87415e7d6f32436ab995a89adac1d033, user_id=4756e4fb7f2d4273938edbec66226d85, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:03.854807 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a078b3-b12b-431b-bdba-0c9b51e7b276 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:03.870481 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1279/2558] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:55:03 2022] PUT /identity/v3/projects/87415e7d6f32436ab995a89adac1d033/users/4756e4fb7f2d4273938edbec66226d85/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:55:03.885529 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e07f6-fa57-4d56-bb79-52f9ee48a6cc None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:03.885812 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e07f6-fa57-4d56-bb79-52f9ee48a6cc None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:03.886104 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e07f6-fa57-4d56-bb79-52f9ee48a6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:04.042643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-a77e07f6-fa57-4d56-bb79-52f9ee48a6cc None None] MFA Rules not processed for user `4756e4fb7f2d4273938edbec66226d85`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:04.045872 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-a77e07f6-fa57-4d56-bb79-52f9ee48a6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:04.138938 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1280/2559] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:03 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:55:04.141585 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a078b3-b12b-431b-bdba-0c9b51e7b276 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:04.183897 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:DELETE=6,keystone:SELECT=88,keystone:INSERT=6 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:04.219659 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a078b3-b12b-431b-bdba-0c9b51e7b276 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:04.224403 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a6152-a6c3-4b72-a55d-6a8b41a87ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:04.225063 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a6152-a6c3-4b72-a55d-6a8b41a87ffb admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:04.225290 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a6152-a6c3-4b72-a55d-6a8b41a87ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:04.225563 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a6152-a6c3-4b72-a55d-6a8b41a87ffb admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:04.226196 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6a6152-a6c3-4b72-a55d-6a8b41a87ffb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:04.228162 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6a6152-a6c3-4b72-a55d-6a8b41a87ffb admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:04.257521 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1280/2560] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:55:04 2022] POST /identity/v3/projects => generated 387 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:55:04.261639 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e07f6-fa57-4d56-bb79-52f9ee48a6cc None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:04.267474 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:04.273696 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e07f6-fa57-4d56-bb79-52f9ee48a6cc None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:04.336064 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec3dd6-ea4c-4428-8e2b-0e930fe63357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:04.336676 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec3dd6-ea4c-4428-8e2b-0e930fe63357 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:04.336972 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec3dd6-ea4c-4428-8e2b-0e930fe63357 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:04.337230 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec3dd6-ea4c-4428-8e2b-0e930fe63357 admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:04.337865 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec3dd6-ea4c-4428-8e2b-0e930fe63357 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:04.339903 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec3dd6-ea4c-4428-8e2b-0e930fe63357 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:04.365169 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1281/2561] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:55:04 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:55:04.366434 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a6152-a6c3-4b72-a55d-6a8b41a87ffb admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:04.376812 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a6152-a6c3-4b72-a55d-6a8b41a87ffb admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:04.382021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961f94d-cdce-40f7-a852-2393a4d69408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:04.382612 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961f94d-cdce-40f7-a852-2393a4d69408 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:04.382960 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961f94d-cdce-40f7-a852-2393a4d69408 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:04.383231 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961f94d-cdce-40f7-a852-2393a4d69408 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:04.383877 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3961f94d-cdce-40f7-a852-2393a4d69408 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:04.385489 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3961f94d-cdce-40f7-a852-2393a4d69408 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:04.395773 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1281/2562] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:55:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:55:04.399554 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec3dd6-ea4c-4428-8e2b-0e930fe63357 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:04.409949 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec3dd6-ea4c-4428-8e2b-0e930fe63357 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:04.414503 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043f830-c590-425b-8f5d-b7c85869d4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:04.415325 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043f830-c590-425b-8f5d-b7c85869d4c3 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:04.415610 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043f830-c590-425b-8f5d-b7c85869d4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:04.415877 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043f830-c590-425b-8f5d-b7c85869d4c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:04.416805 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a043f830-c590-425b-8f5d-b7c85869d4c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:04.418383 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a043f830-c590-425b-8f5d-b7c85869d4c3 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:04.429638 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1282/2563] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:55:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:55:04.433511 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961f94d-cdce-40f7-a852-2393a4d69408 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:04.444262 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961f94d-cdce-40f7-a852-2393a4d69408 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:04.449428 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb8718-8729-4867-9093-ed57be0419eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:04.450087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb8718-8729-4867-9093-ed57be0419eb admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:04.450367 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb8718-8729-4867-9093-ed57be0419eb admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:04.450664 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb8718-8729-4867-9093-ed57be0419eb admin admin] PATH_INFO: `/v3/projects/b58d80ca9ff843018b53a9911cf974e1/users/0560d7db956142daa4c13577e2e00f88/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:04.466348 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb8718-8729-4867-9093-ed57be0419eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b58d80ca9ff843018b53a9911cf974e1, user_id=0560d7db956142daa4c13577e2e00f88, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:04.469091 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb8718-8729-4867-9093-ed57be0419eb admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:04.482368 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1282/2564] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:55:04 2022] PUT /identity/v3/projects/b58d80ca9ff843018b53a9911cf974e1/users/0560d7db956142daa4c13577e2e00f88/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:04.493300 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088887dc-21dd-43bf-be9c-cc7d0d63df21 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:04.493786 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088887dc-21dd-43bf-be9c-cc7d0d63df21 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:04.493898 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088887dc-21dd-43bf-be9c-cc7d0d63df21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:04.521485 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-088887dc-21dd-43bf-be9c-cc7d0d63df21 None None] MFA Rules not processed for user `0560d7db956142daa4c13577e2e00f88`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:04.522760 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-088887dc-21dd-43bf-be9c-cc7d0d63df21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:04.588157 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1283/2565] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:04 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:55:04.592821 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb8718-8729-4867-9093-ed57be0419eb admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:04.655932 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb8718-8729-4867-9093-ed57be0419eb admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:04.661087 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd806d-4c88-4a11-9733-f2bf56e908cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:04.661682 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd806d-4c88-4a11-9733-f2bf56e908cd admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:04.661958 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd806d-4c88-4a11-9733-f2bf56e908cd admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:04.662226 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd806d-4c88-4a11-9733-f2bf56e908cd admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:04.662851 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efd806d-4c88-4a11-9733-f2bf56e908cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:04.664775 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efd806d-4c88-4a11-9733-f2bf56e908cd admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:04.690584 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1283/2566] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:55:04 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:55:04.694878 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088887dc-21dd-43bf-be9c-cc7d0d63df21 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:04.705307 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088887dc-21dd-43bf-be9c-cc7d0d63df21 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:04.759822 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564a397-13aa-4d93-a758-4442bc4e7b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:04.760384 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564a397-13aa-4d93-a758-4442bc4e7b49 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:04.760660 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564a397-13aa-4d93-a758-4442bc4e7b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:04.760925 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564a397-13aa-4d93-a758-4442bc4e7b49 admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:04.761543 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5564a397-13aa-4d93-a758-4442bc4e7b49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:04.763454 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5564a397-13aa-4d93-a758-4442bc4e7b49 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:04.780544 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:04.789064 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1284/2567] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:55:04 2022] POST /identity/v3/users => generated 299 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:55:04.790325 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd806d-4c88-4a11-9733-f2bf56e908cd admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:04.800845 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd806d-4c88-4a11-9733-f2bf56e908cd admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:04.805492 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c73d1-7b47-4b81-a1c5-8c07c15c32a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:04.806083 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c73d1-7b47-4b81-a1c5-8c07c15c32a3 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:04.806363 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c73d1-7b47-4b81-a1c5-8c07c15c32a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:04.806636 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c73d1-7b47-4b81-a1c5-8c07c15c32a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:04.807277 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1c73d1-7b47-4b81-a1c5-8c07c15c32a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:04.808904 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1c73d1-7b47-4b81-a1c5-8c07c15c32a3 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:04.819205 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1284/2568] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:55:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:55:04.822948 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564a397-13aa-4d93-a758-4442bc4e7b49 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:04.833075 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564a397-13aa-4d93-a758-4442bc4e7b49 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:04.837596 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aed196b-0208-422c-9f09-a4ef74f2d343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:04.838226 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed196b-0208-422c-9f09-a4ef74f2d343 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:04.838501 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed196b-0208-422c-9f09-a4ef74f2d343 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:04.838770 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed196b-0208-422c-9f09-a4ef74f2d343 admin admin] PATH_INFO: `/v3/projects/aaa45674002f4c8791bb9ca080bc412c/users/4a97cfe03737441db8818a35e6b1fd17/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:04.853389 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aed196b-0208-422c-9f09-a4ef74f2d343 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaa45674002f4c8791bb9ca080bc412c, user_id=4a97cfe03737441db8818a35e6b1fd17, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:04.856045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aed196b-0208-422c-9f09-a4ef74f2d343 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:04.869591 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1285/2569] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:55:04 2022] PUT /identity/v3/projects/aaa45674002f4c8791bb9ca080bc412c/users/4a97cfe03737441db8818a35e6b1fd17/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:04.873624 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c73d1-7b47-4b81-a1c5-8c07c15c32a3 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:04.883483 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c73d1-7b47-4b81-a1c5-8c07c15c32a3 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:04.930977 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:04.940917 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2298963-5b76-4152-ab0e-224f237284ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:04.941518 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2298963-5b76-4152-ab0e-224f237284ca admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:04.941793 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2298963-5b76-4152-ab0e-224f237284ca admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:04.942063 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2298963-5b76-4152-ab0e-224f237284ca admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:04.942696 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2298963-5b76-4152-ab0e-224f237284ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:04.944346 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2298963-5b76-4152-ab0e-224f237284ca admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:04.958741 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1285/2570] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:55:04 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:55:04.960063 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aed196b-0208-422c-9f09-a4ef74f2d343 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:04.970833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aed196b-0208-422c-9f09-a4ef74f2d343 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:04.976060 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55cf2b2-3aa9-4ec3-a48a-5330fa76ab75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:04.976703 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cf2b2-3aa9-4ec3-a48a-5330fa76ab75 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:04.976985 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cf2b2-3aa9-4ec3-a48a-5330fa76ab75 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:04.977613 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cf2b2-3aa9-4ec3-a48a-5330fa76ab75 admin admin] PATH_INFO: `/v3/projects/aaa45674002f4c8791bb9ca080bc412c/users/4a97cfe03737441db8818a35e6b1fd17/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:04.979525 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55cf2b2-3aa9-4ec3-a48a-5330fa76ab75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaa45674002f4c8791bb9ca080bc412c, user_id=4a97cfe03737441db8818a35e6b1fd17, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:04.982209 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55cf2b2-3aa9-4ec3-a48a-5330fa76ab75 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:04.992456 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1286/2571] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:55:04 2022] PUT /identity/v3/projects/aaa45674002f4c8791bb9ca080bc412c/users/4a97cfe03737441db8818a35e6b1fd17/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:05.003881 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94cef4-d242-498d-8b5f-c8dbd193d000 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:05.004263 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94cef4-d242-498d-8b5f-c8dbd193d000 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:05.004564 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94cef4-d242-498d-8b5f-c8dbd193d000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:05.033421 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-7f94cef4-d242-498d-8b5f-c8dbd193d000 None None] MFA Rules not processed for user `4a97cfe03737441db8818a35e6b1fd17`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:05.035244 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-7f94cef4-d242-498d-8b5f-c8dbd193d000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:05.101762 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1286/2572] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:05 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:55:05.112628 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55cf2b2-3aa9-4ec3-a48a-5330fa76ab75 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:05.114313 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-b55cf2b2-3aa9-4ec3-a48a-5330fa76ab75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:05.170676 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55cf2b2-3aa9-4ec3-a48a-5330fa76ab75 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:05.174309 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d9b08-c2d0-47f3-9a60-fd83ccedcc43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:05.174912 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d9b08-c2d0-47f3-9a60-fd83ccedcc43 service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:05.175187 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d9b08-c2d0-47f3-9a60-fd83ccedcc43 service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:05.175459 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d9b08-c2d0-47f3-9a60-fd83ccedcc43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:05.184560 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d9b08-c2d0-47f3-9a60-fd83ccedcc43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:05.186005 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d9b08-c2d0-47f3-9a60-fd83ccedcc43 service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:05.195905 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1287/2573] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:55:05 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:55:05.796941 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f94cef4-d242-498d-8b5f-c8dbd193d000 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:05.806858 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f94cef4-d242-498d-8b5f-c8dbd193d000 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:05.810393 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84814dd3-2d64-4a3f-9c88-821e58fdfd9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:05.810990 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84814dd3-2d64-4a3f-9c88-821e58fdfd9d service cinder] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:05.811267 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84814dd3-2d64-4a3f-9c88-821e58fdfd9d service cinder] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:05.811539 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84814dd3-2d64-4a3f-9c88-821e58fdfd9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:05.866892 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84814dd3-2d64-4a3f-9c88-821e58fdfd9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:05.868658 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84814dd3-2d64-4a3f-9c88-821e58fdfd9d service cinder] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:05.877529 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1287/2574] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:55:05 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:55:15.191707 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=41,keystone:INSERT=2 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:15.258337 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d9b08-c2d0-47f3-9a60-fd83ccedcc43 service cinder] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:15.269558 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d9b08-c2d0-47f3-9a60-fd83ccedcc43 service cinder] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:15.273725 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8759a5-bf82-4017-bfdb-9cd89a2b013f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:15.274500 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8759a5-bf82-4017-bfdb-9cd89a2b013f service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:15.275195 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8759a5-bf82-4017-bfdb-9cd89a2b013f service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:15.275480 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8759a5-bf82-4017-bfdb-9cd89a2b013f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:15.348693 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8759a5-bf82-4017-bfdb-9cd89a2b013f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:15.350158 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8759a5-bf82-4017-bfdb-9cd89a2b013f service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:15.364060 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1288/2575] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:55:15 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:55:15.873579 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=67 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:16.777730 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84814dd3-2d64-4a3f-9c88-821e58fdfd9d service cinder] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:16.779264 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-84814dd3-2d64-4a3f-9c88-821e58fdfd9d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:16.848698 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84814dd3-2d64-4a3f-9c88-821e58fdfd9d service cinder] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:16.854189 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9fbf8-83e9-4fb5-9683-b63bde2f7a30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:16.854839 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9fbf8-83e9-4fb5-9683-b63bde2f7a30 service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:16.855091 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9fbf8-83e9-4fb5-9683-b63bde2f7a30 service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:16.855359 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9fbf8-83e9-4fb5-9683-b63bde2f7a30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:16.857301 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-00d9fbf8-83e9-4fb5-9683-b63bde2f7a30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:16.940216 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9fbf8-83e9-4fb5-9683-b63bde2f7a30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:16.941840 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9fbf8-83e9-4fb5-9683-b63bde2f7a30 service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:16.952708 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1288/2576] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:55:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:55:17.702146 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8759a5-bf82-4017-bfdb-9cd89a2b013f service cinder] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:17.713678 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8759a5-bf82-4017-bfdb-9cd89a2b013f service cinder] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:17.718352 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e20ab-570c-4a24-8bdf-f509d3ccf9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:17.719004 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e20ab-570c-4a24-8bdf-f509d3ccf9ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:17.719312 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e20ab-570c-4a24-8bdf-f509d3ccf9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:17.719583 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e20ab-570c-4a24-8bdf-f509d3ccf9ed admin admin] PATH_INFO: `/v3/users/4756e4fb7f2d4273938edbec66226d85` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:17.720941 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9e20ab-570c-4a24-8bdf-f509d3ccf9ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=4756e4fb7f2d4273938edbec66226d85)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:17.723727 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9e20ab-570c-4a24-8bdf-f509d3ccf9ed admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:17.772678 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-0c9e20ab-570c-4a24-8bdf-f509d3ccf9ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4756e4fb7f2d4273938edbec66226d85'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:17.773420 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-0c9e20ab-570c-4a24-8bdf-f509d3ccf9ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4756e4fb7f2d4273938edbec66226d85'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:17.788158 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-0c9e20ab-570c-4a24-8bdf-f509d3ccf9ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4756e4fb7f2d4273938edbec66226d85'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:17.795788 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-0c9e20ab-570c-4a24-8bdf-f509d3ccf9ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4756e4fb7f2d4273938edbec66226d85'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:17.796439 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-0c9e20ab-570c-4a24-8bdf-f509d3ccf9ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4756e4fb7f2d4273938edbec66226d85'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:17.809840 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1289/2577] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:55:17 2022] DELETE /identity/v3/users/4756e4fb7f2d4273938edbec66226d85 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:17.814447 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020f06c-9b49-434a-a30d-9355be5cbab2 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:17.814935 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020f06c-9b49-434a-a30d-9355be5cbab2 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:17.815519 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020f06c-9b49-434a-a30d-9355be5cbab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:17.835772 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-2020f06c-9b49-434a-a30d-9355be5cbab2 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:17.837108 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-2020f06c-9b49-434a-a30d-9355be5cbab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:17.908855 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1289/2578] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:55:17.920484 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e20ab-570c-4a24-8bdf-f509d3ccf9ed admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:17.921979 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-0c9e20ab-570c-4a24-8bdf-f509d3ccf9ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:18.021106 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e20ab-570c-4a24-8bdf-f509d3ccf9ed admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:18.025464 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fffcda-b095-46fb-add6-cad1a236605e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:18.026492 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fffcda-b095-46fb-add6-cad1a236605e service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:18.027006 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fffcda-b095-46fb-add6-cad1a236605e service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:18.027613 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fffcda-b095-46fb-add6-cad1a236605e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:18.040919 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fffcda-b095-46fb-add6-cad1a236605e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:18.042671 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fffcda-b095-46fb-add6-cad1a236605e service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:18.055932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1290/2579] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:55:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 23:55:18.787312 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2020f06c-9b49-434a-a30d-9355be5cbab2 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:18.793173 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-2020f06c-9b49-434a-a30d-9355be5cbab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:18.808983 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2020f06c-9b49-434a-a30d-9355be5cbab2 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:18.813725 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e346ae04-8550-4434-b7af-69910e308d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:18.814355 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346ae04-8550-4434-b7af-69910e308d8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:18.814860 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346ae04-8550-4434-b7af-69910e308d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:18.815136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346ae04-8550-4434-b7af-69910e308d8a admin admin] PATH_INFO: `/v3/projects/87415e7d6f32436ab995a89adac1d033` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:18.816341 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e346ae04-8550-4434-b7af-69910e308d8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=87415e7d6f32436ab995a89adac1d033)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:18.818280 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e346ae04-8550-4434-b7af-69910e308d8a admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:18.872868 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:18.877039 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-e346ae04-8550-4434-b7af-69910e308d8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87415e7d6f32436ab995a89adac1d033', 'request_id': 'req-e346ae04-8550-4434-b7af-69910e308d8a'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:18.887293 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-e346ae04-8550-4434-b7af-69910e308d8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87415e7d6f32436ab995a89adac1d033', 'request_id': 'req-e346ae04-8550-4434-b7af-69910e308d8a'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:18.897927 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-e346ae04-8550-4434-b7af-69910e308d8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87415e7d6f32436ab995a89adac1d033', 'request_id': 'req-e346ae04-8550-4434-b7af-69910e308d8a'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:18.914846 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-e346ae04-8550-4434-b7af-69910e308d8a admin admin] The token cache is being invalidate because project 87415e7d6f32436ab995a89adac1d033 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:55:18.915144 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-e346ae04-8550-4434-b7af-69910e308d8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:18.915803 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-e346ae04-8550-4434-b7af-69910e308d8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:18.920621 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1290/2580] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:55:18 2022] DELETE /identity/v3/projects/87415e7d6f32436ab995a89adac1d033 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:18.922134 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fffcda-b095-46fb-add6-cad1a236605e service neutron] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:18.932820 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fffcda-b095-46fb-add6-cad1a236605e service neutron] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:18.942877 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:19.013840 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f104819a-461a-452d-9193-314fc6416b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:19.015041 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104819a-461a-452d-9193-314fc6416b98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:19.015326 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104819a-461a-452d-9193-314fc6416b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:19.015595 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104819a-461a-452d-9193-314fc6416b98 admin admin] PATH_INFO: `/v3/users/0560d7db956142daa4c13577e2e00f88` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:19.018305 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f104819a-461a-452d-9193-314fc6416b98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0560d7db956142daa4c13577e2e00f88)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:19.022792 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f104819a-461a-452d-9193-314fc6416b98 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:19.101003 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-f104819a-461a-452d-9193-314fc6416b98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0560d7db956142daa4c13577e2e00f88'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:19.107484 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-f104819a-461a-452d-9193-314fc6416b98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0560d7db956142daa4c13577e2e00f88'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:19.122380 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-f104819a-461a-452d-9193-314fc6416b98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0560d7db956142daa4c13577e2e00f88'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:19.133375 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-f104819a-461a-452d-9193-314fc6416b98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0560d7db956142daa4c13577e2e00f88'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:19.134320 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-f104819a-461a-452d-9193-314fc6416b98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0560d7db956142daa4c13577e2e00f88'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:19.152181 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1291/2581] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:55:18 2022] DELETE /identity/v3/users/0560d7db956142daa4c13577e2e00f88 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:19.476171 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e346ae04-8550-4434-b7af-69910e308d8a admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:19.477882 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-e346ae04-8550-4434-b7af-69910e308d8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:19.494123 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e346ae04-8550-4434-b7af-69910e308d8a admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:19.560615 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8fd7f8-adae-4df0-a334-7e87444ba760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:19.561294 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8fd7f8-adae-4df0-a334-7e87444ba760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:19.561574 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8fd7f8-adae-4df0-a334-7e87444ba760 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:19.561853 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8fd7f8-adae-4df0-a334-7e87444ba760 admin admin] PATH_INFO: `/v3/projects/b58d80ca9ff843018b53a9911cf974e1` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:19.563012 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8fd7f8-adae-4df0-a334-7e87444ba760 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b58d80ca9ff843018b53a9911cf974e1)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:19.565381 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8fd7f8-adae-4df0-a334-7e87444ba760 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:19.617540 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-2f8fd7f8-adae-4df0-a334-7e87444ba760 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b58d80ca9ff843018b53a9911cf974e1', 'request_id': 'req-2f8fd7f8-adae-4df0-a334-7e87444ba760'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:19.623506 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-2f8fd7f8-adae-4df0-a334-7e87444ba760 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b58d80ca9ff843018b53a9911cf974e1', 'request_id': 'req-2f8fd7f8-adae-4df0-a334-7e87444ba760'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:19.633774 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-2f8fd7f8-adae-4df0-a334-7e87444ba760 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b58d80ca9ff843018b53a9911cf974e1', 'request_id': 'req-2f8fd7f8-adae-4df0-a334-7e87444ba760'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:19.650187 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-2f8fd7f8-adae-4df0-a334-7e87444ba760 admin admin] The token cache is being invalidate because project b58d80ca9ff843018b53a9911cf974e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:55:19.652660 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-2f8fd7f8-adae-4df0-a334-7e87444ba760 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:19.653608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-2f8fd7f8-adae-4df0-a334-7e87444ba760 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:19.655284 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1291/2582] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:55:19 2022] DELETE /identity/v3/projects/b58d80ca9ff843018b53a9911cf974e1 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:19.659562 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f104819a-461a-452d-9193-314fc6416b98 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:19.661053 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-f104819a-461a-452d-9193-314fc6416b98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:19.691459 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f104819a-461a-452d-9193-314fc6416b98 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:19.775284 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8fd7f8-adae-4df0-a334-7e87444ba760 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:19.776794 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-2f8fd7f8-adae-4df0-a334-7e87444ba760 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:19.787020 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf69ed1-0587-4291-973c-533ec153d2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:19.787986 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf69ed1-0587-4291-973c-533ec153d2de admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:19.788511 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf69ed1-0587-4291-973c-533ec153d2de admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:19.788957 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf69ed1-0587-4291-973c-533ec153d2de admin admin] PATH_INFO: `/v3/users/4a97cfe03737441db8818a35e6b1fd17` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:19.790381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf69ed1-0587-4291-973c-533ec153d2de admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a97cfe03737441db8818a35e6b1fd17)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:19.792917 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf69ed1-0587-4291-973c-533ec153d2de admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:19.844133 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:19.855076 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8fd7f8-adae-4df0-a334-7e87444ba760 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:19.860275 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-bdf69ed1-0587-4291-973c-533ec153d2de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a97cfe03737441db8818a35e6b1fd17'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:19.860445 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d769238-6a06-48e5-8de2-be216d8b2ab2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:19.861199 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d769238-6a06-48e5-8de2-be216d8b2ab2 service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:19.861585 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d769238-6a06-48e5-8de2-be216d8b2ab2 service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:19.861863 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-bdf69ed1-0587-4291-973c-533ec153d2de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a97cfe03737441db8818a35e6b1fd17'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:19.862297 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d769238-6a06-48e5-8de2-be216d8b2ab2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:19.865210 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-0d769238-6a06-48e5-8de2-be216d8b2ab2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:19.879339 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-bdf69ed1-0587-4291-973c-533ec153d2de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a97cfe03737441db8818a35e6b1fd17'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:19.879550 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d769238-6a06-48e5-8de2-be216d8b2ab2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:19.880892 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d769238-6a06-48e5-8de2-be216d8b2ab2 service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:19.888429 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-bdf69ed1-0587-4291-973c-533ec153d2de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a97cfe03737441db8818a35e6b1fd17'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:19.889332 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-bdf69ed1-0587-4291-973c-533ec153d2de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a97cfe03737441db8818a35e6b1fd17'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:19.892272 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1292/2583] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:55:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:55:19.909059 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1292/2584] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:55:19 2022] DELETE /identity/v3/users/4a97cfe03737441db8818a35e6b1fd17 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:55:20.614784 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d769238-6a06-48e5-8de2-be216d8b2ab2 service neutron] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:20.616539 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-0d769238-6a06-48e5-8de2-be216d8b2ab2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:20.651072 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d769238-6a06-48e5-8de2-be216d8b2ab2 service neutron] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:20.664617 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:20.795074 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f92563-9fc9-42df-8ce8-99f3421489e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:20.795719 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f92563-9fc9-42df-8ce8-99f3421489e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:20.796478 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f92563-9fc9-42df-8ce8-99f3421489e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:20.796766 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f92563-9fc9-42df-8ce8-99f3421489e5 admin admin] PATH_INFO: `/v3/projects/aaa45674002f4c8791bb9ca080bc412c` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:20.798880 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f92563-9fc9-42df-8ce8-99f3421489e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaa45674002f4c8791bb9ca080bc412c)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:20.803084 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f92563-9fc9-42df-8ce8-99f3421489e5 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:20.896674 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-b7f92563-9fc9-42df-8ce8-99f3421489e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaa45674002f4c8791bb9ca080bc412c', 'request_id': 'req-b7f92563-9fc9-42df-8ce8-99f3421489e5'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:20.904167 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-b7f92563-9fc9-42df-8ce8-99f3421489e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaa45674002f4c8791bb9ca080bc412c', 'request_id': 'req-b7f92563-9fc9-42df-8ce8-99f3421489e5'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:20.917865 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-b7f92563-9fc9-42df-8ce8-99f3421489e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaa45674002f4c8791bb9ca080bc412c', 'request_id': 'req-b7f92563-9fc9-42df-8ce8-99f3421489e5'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:20.942135 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-b7f92563-9fc9-42df-8ce8-99f3421489e5 admin admin] The token cache is being invalidate because project aaa45674002f4c8791bb9ca080bc412c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:55:20.944354 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-b7f92563-9fc9-42df-8ce8-99f3421489e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:20.945374 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-b7f92563-9fc9-42df-8ce8-99f3421489e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:20.948966 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1293/2585] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:55:20 2022] DELETE /identity/v3/projects/aaa45674002f4c8791bb9ca080bc412c => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:20.961055 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7801d26-a038-46a0-8511-a67ebdc2477c None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:20.961589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7801d26-a038-46a0-8511-a67ebdc2477c None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:20.962930 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7801d26-a038-46a0-8511-a67ebdc2477c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:20.991418 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-b7801d26-a038-46a0-8511-a67ebdc2477c None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:20.992710 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-b7801d26-a038-46a0-8511-a67ebdc2477c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:21.153469 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f92563-9fc9-42df-8ce8-99f3421489e5 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:21.157306 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-b7f92563-9fc9-42df-8ce8-99f3421489e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:21.180759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1293/2586] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:55:21.181491 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f92563-9fc9-42df-8ce8-99f3421489e5 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:21.183179 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed236b2-1256-4664-8dd7-07661c6868bc None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:21.183538 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed236b2-1256-4664-8dd7-07661c6868bc None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:21.183955 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed236b2-1256-4664-8dd7-07661c6868bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:21.195688 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b3d2a2-8bf3-47ab-a3f1-9fb6e998fba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:21.197883 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3d2a2-8bf3-47ab-a3f1-9fb6e998fba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:21.198237 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3d2a2-8bf3-47ab-a3f1-9fb6e998fba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:21.198583 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3d2a2-8bf3-47ab-a3f1-9fb6e998fba5 admin admin] PATH_INFO: `/v3/users/2da686247e7f490ab7924d0456da60d5` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:21.201124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b3d2a2-8bf3-47ab-a3f1-9fb6e998fba5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2da686247e7f490ab7924d0456da60d5)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:21.204222 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b3d2a2-8bf3-47ab-a3f1-9fb6e998fba5 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:21.233193 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-4ed236b2-1256-4664-8dd7-07661c6868bc None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:21.234551 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-4ed236b2-1256-4664-8dd7-07661c6868bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:21.245826 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1294/2587] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:55:21.249912 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed236b2-1256-4664-8dd7-07661c6868bc None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:21.263076 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed236b2-1256-4664-8dd7-07661c6868bc None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:21.273738 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6805330-f8de-4e4d-940e-11e10645ab9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:21.275424 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6805330-f8de-4e4d-940e-11e10645ab9e admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:21.276597 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6805330-f8de-4e4d-940e-11e10645ab9e admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:21.277364 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6805330-f8de-4e4d-940e-11e10645ab9e admin admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:21.279590 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6805330-f8de-4e4d-940e-11e10645ab9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:55:21.280059 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6805330-f8de-4e4d-940e-11e10645ab9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:21.282087 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6805330-f8de-4e4d-940e-11e10645ab9e admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:21.304134 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-62b3d2a2-8bf3-47ab-a3f1-9fb6e998fba5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2da686247e7f490ab7924d0456da60d5'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:21.305642 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-62b3d2a2-8bf3-47ab-a3f1-9fb6e998fba5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2da686247e7f490ab7924d0456da60d5'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:21.306551 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-62b3d2a2-8bf3-47ab-a3f1-9fb6e998fba5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2da686247e7f490ab7924d0456da60d5'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:21.312506 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1295/2588] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:55:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:55:21.314585 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6805330-f8de-4e4d-940e-11e10645ab9e admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:21.317607 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-f6805330-f8de-4e4d-940e-11e10645ab9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:21.322197 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-62b3d2a2-8bf3-47ab-a3f1-9fb6e998fba5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2da686247e7f490ab7924d0456da60d5'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:21.333258 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-62b3d2a2-8bf3-47ab-a3f1-9fb6e998fba5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2da686247e7f490ab7924d0456da60d5'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:21.338697 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6805330-f8de-4e4d-940e-11e10645ab9e admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:21.345157 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e84879-1bfc-4db2-8b4d-d5c56d13519e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:21.345389 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e84879-1bfc-4db2-8b4d-d5c56d13519e admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:21.345742 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e84879-1bfc-4db2-8b4d-d5c56d13519e admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:21.346105 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e84879-1bfc-4db2-8b4d-d5c56d13519e admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:21.346807 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e84879-1bfc-4db2-8b4d-d5c56d13519e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:21.349531 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e84879-1bfc-4db2-8b4d-d5c56d13519e admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:21.366627 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1294/2589] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:55:21 2022] DELETE /identity/v3/users/2da686247e7f490ab7924d0456da60d5 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:21.384018 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1296/2590] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:55:21 2022] POST /identity/v3/projects => generated 375 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:55:21.388319 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b3d2a2-8bf3-47ab-a3f1-9fb6e998fba5 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:21.400190 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:21.477297 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b3d2a2-8bf3-47ab-a3f1-9fb6e998fba5 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:21.541866 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4d22c8-dbfc-4beb-858e-70660e43fd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:21.542670 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d22c8-dbfc-4beb-858e-70660e43fd31 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:21.542969 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d22c8-dbfc-4beb-858e-70660e43fd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:21.543216 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d22c8-dbfc-4beb-858e-70660e43fd31 admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:21.544005 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4d22c8-dbfc-4beb-858e-70660e43fd31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:21.546291 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4d22c8-dbfc-4beb-858e-70660e43fd31 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:21.567193 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e84879-1bfc-4db2-8b4d-d5c56d13519e admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:21.569411 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-e6e84879-1bfc-4db2-8b4d-d5c56d13519e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:21.582605 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1295/2591] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:55:21 2022] POST /identity/v3/users => generated 288 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:55:21.584190 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4d22c8-dbfc-4beb-858e-70660e43fd31 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:21.592800 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e84879-1bfc-4db2-8b4d-d5c56d13519e admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:21.602196 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4d22c8-dbfc-4beb-858e-70660e43fd31 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:21.602699 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9bd15d-7ca4-45d8-9d5c-6b45ed4999b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:21.604135 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9bd15d-7ca4-45d8-9d5c-6b45ed4999b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:21.604761 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9bd15d-7ca4-45d8-9d5c-6b45ed4999b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:21.605176 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9bd15d-7ca4-45d8-9d5c-6b45ed4999b5 admin admin] PATH_INFO: `/v3/projects/d9574c6bc0ea4f78a0145fc2764a6fa9` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:21.606866 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9bd15d-7ca4-45d8-9d5c-6b45ed4999b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9574c6bc0ea4f78a0145fc2764a6fa9)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:21.610020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c92a06e-70c7-472e-b562-680d3109599a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:21.610724 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92a06e-70c7-472e-b562-680d3109599a admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:21.611057 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9bd15d-7ca4-45d8-9d5c-6b45ed4999b5 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:21.611225 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92a06e-70c7-472e-b562-680d3109599a admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:21.611501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92a06e-70c7-472e-b562-680d3109599a admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:21.612788 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c92a06e-70c7-472e-b562-680d3109599a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:21.614542 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c92a06e-70c7-472e-b562-680d3109599a admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:21.627372 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1296/2592] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:55:21 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 23:55:21.631930 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c92a06e-70c7-472e-b562-680d3109599a admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:21.644902 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c92a06e-70c7-472e-b562-680d3109599a admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:21.650038 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0373beb3-4a8c-4959-92f8-5c8aeb160f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:21.650729 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0373beb3-4a8c-4959-92f8-5c8aeb160f22 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:21.651253 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0373beb3-4a8c-4959-92f8-5c8aeb160f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:21.651504 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0373beb3-4a8c-4959-92f8-5c8aeb160f22 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:21.652602 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0373beb3-4a8c-4959-92f8-5c8aeb160f22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:21.654612 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0373beb3-4a8c-4959-92f8-5c8aeb160f22 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:21.671744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1297/2593] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:55:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 23:55:21.673603 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0373beb3-4a8c-4959-92f8-5c8aeb160f22 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:21.682493 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-9f9bd15d-7ca4-45d8-9d5c-6b45ed4999b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9574c6bc0ea4f78a0145fc2764a6fa9', 'request_id': 'req-9f9bd15d-7ca4-45d8-9d5c-6b45ed4999b5'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:21.693371 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0373beb3-4a8c-4959-92f8-5c8aeb160f22 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:21.697146 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-9f9bd15d-7ca4-45d8-9d5c-6b45ed4999b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9574c6bc0ea4f78a0145fc2764a6fa9', 'request_id': 'req-9f9bd15d-7ca4-45d8-9d5c-6b45ed4999b5'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:21.707720 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-9f9bd15d-7ca4-45d8-9d5c-6b45ed4999b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9574c6bc0ea4f78a0145fc2764a6fa9', 'request_id': 'req-9f9bd15d-7ca4-45d8-9d5c-6b45ed4999b5'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:21.710648 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:21.730744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-9f9bd15d-7ca4-45d8-9d5c-6b45ed4999b5 admin admin] The token cache is being invalidate because project d9574c6bc0ea4f78a0145fc2764a6fa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:55:21.731571 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-9f9bd15d-7ca4-45d8-9d5c-6b45ed4999b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:21.732728 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-9f9bd15d-7ca4-45d8-9d5c-6b45ed4999b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:21.734868 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1297/2594] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:55:21 2022] DELETE /identity/v3/projects/d9574c6bc0ea4f78a0145fc2764a6fa9 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:55:21.739733 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf792d-3a43-41fc-bcdb-87c9a456da1f None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:21.740062 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf792d-3a43-41fc-bcdb-87c9a456da1f None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:21.740708 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf792d-3a43-41fc-bcdb-87c9a456da1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:21.779438 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a264a6fc-2c25-48cc-a7a9-43e6723ea4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:21.781825 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264a6fc-2c25-48cc-a7a9-43e6723ea4b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:21.785420 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264a6fc-2c25-48cc-a7a9-43e6723ea4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:21.785420 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264a6fc-2c25-48cc-a7a9-43e6723ea4b9 admin admin] PATH_INFO: `/v3/projects/cab6ab8789e34c50b026a490df2f3a36/users/90fc6085547a42099a8e6dbf032290f5/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:21.790133 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-98cf792d-3a43-41fc-bcdb-87c9a456da1f None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:21.794569 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-98cf792d-3a43-41fc-bcdb-87c9a456da1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:21.813909 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1298/2595] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:55:21.821515 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a264a6fc-2c25-48cc-a7a9-43e6723ea4b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cab6ab8789e34c50b026a490df2f3a36, user_id=90fc6085547a42099a8e6dbf032290f5, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:21.821515 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a264a6fc-2c25-48cc-a7a9-43e6723ea4b9 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:21.822957 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18abae35-5d00-4bc4-90b9-288cb8b1e5be None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:21.823233 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18abae35-5d00-4bc4-90b9-288cb8b1e5be None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:21.823498 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18abae35-5d00-4bc4-90b9-288cb8b1e5be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:21.852789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1298/2596] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:55:21 2022] PUT /identity/v3/projects/cab6ab8789e34c50b026a490df2f3a36/users/90fc6085547a42099a8e6dbf032290f5/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:21.868199 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bba57a-e926-40d3-ac29-f1e531d27e4e None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:21.868357 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bba57a-e926-40d3-ac29-f1e531d27e4e None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:21.868616 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bba57a-e926-40d3-ac29-f1e531d27e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:21.875234 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-18abae35-5d00-4bc4-90b9-288cb8b1e5be None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:21.876476 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-18abae35-5d00-4bc4-90b9-288cb8b1e5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:21.893668 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:21.927208 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-e2bba57a-e926-40d3-ac29-f1e531d27e4e None None] MFA Rules not processed for user `90fc6085547a42099a8e6dbf032290f5`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:21.928384 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-e2bba57a-e926-40d3-ac29-f1e531d27e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:21.973402 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1299/2597] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:55:21.978659 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18abae35-5d00-4bc4-90b9-288cb8b1e5be None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:21.991011 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18abae35-5d00-4bc4-90b9-288cb8b1e5be None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:21.995856 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e92a23-23ef-4a64-9717-9ec6c01b7e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:21.996575 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e92a23-23ef-4a64-9717-9ec6c01b7e8b admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:21.997260 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e92a23-23ef-4a64-9717-9ec6c01b7e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:21.997740 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e92a23-23ef-4a64-9717-9ec6c01b7e8b admin admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:21.998695 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e92a23-23ef-4a64-9717-9ec6c01b7e8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:55:21.999497 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e92a23-23ef-4a64-9717-9ec6c01b7e8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:22.001396 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e92a23-23ef-4a64-9717-9ec6c01b7e8b admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:22.018871 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1299/2598] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:55:22.025243 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1300/2599] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:55:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:55:22.026137 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bba57a-e926-40d3-ac29-f1e531d27e4e None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:22.032366 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e92a23-23ef-4a64-9717-9ec6c01b7e8b admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:22.034282 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-d9e92a23-23ef-4a64-9717-9ec6c01b7e8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:22.043019 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bba57a-e926-40d3-ac29-f1e531d27e4e None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:22.050953 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8beaf0c-a198-4b35-a9d2-a2dc05e2a3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:22.051895 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8beaf0c-a198-4b35-a9d2-a2dc05e2a3d5 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:22.052220 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8beaf0c-a198-4b35-a9d2-a2dc05e2a3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:22.052544 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8beaf0c-a198-4b35-a9d2-a2dc05e2a3d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:22.053188 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8beaf0c-a198-4b35-a9d2-a2dc05e2a3d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:22.055119 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8beaf0c-a198-4b35-a9d2-a2dc05e2a3d5 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:22.100126 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1300/2600] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:55:22 2022] POST /identity/v3/projects => generated 375 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:55:22.102110 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8beaf0c-a198-4b35-a9d2-a2dc05e2a3d5 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:22.114506 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e92a23-23ef-4a64-9717-9ec6c01b7e8b admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:22.198416 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd0c37a-185c-4602-bcb8-69a17b79496c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:22.200183 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd0c37a-185c-4602-bcb8-69a17b79496c service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:22.200183 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd0c37a-185c-4602-bcb8-69a17b79496c service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:22.200183 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd0c37a-185c-4602-bcb8-69a17b79496c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:22.211092 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8beaf0c-a198-4b35-a9d2-a2dc05e2a3d5 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:22.215989 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dda62a-fbe3-4d26-8b8b-d6d55aeae8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:22.216748 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dda62a-fbe3-4d26-8b8b-d6d55aeae8e3 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:22.217499 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dda62a-fbe3-4d26-8b8b-d6d55aeae8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:22.217499 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dda62a-fbe3-4d26-8b8b-d6d55aeae8e3 admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:22.218246 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dda62a-fbe3-4d26-8b8b-d6d55aeae8e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:22.220539 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dda62a-fbe3-4d26-8b8b-d6d55aeae8e3 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:22.250143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1301/2601] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:55:22 2022] POST /identity/v3/users => generated 288 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:55:22.257515 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dda62a-fbe3-4d26-8b8b-d6d55aeae8e3 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:22.270179 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dda62a-fbe3-4d26-8b8b-d6d55aeae8e3 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:22.276431 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102bfd8-4903-4243-a70a-c45e846f192e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:22.277258 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102bfd8-4903-4243-a70a-c45e846f192e admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:22.277581 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102bfd8-4903-4243-a70a-c45e846f192e admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:22.277927 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102bfd8-4903-4243-a70a-c45e846f192e admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:22.278613 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c102bfd8-4903-4243-a70a-c45e846f192e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:22.280250 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c102bfd8-4903-4243-a70a-c45e846f192e admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:22.282287 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:22.290913 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1302/2602] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 23:55:22.295071 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102bfd8-4903-4243-a70a-c45e846f192e admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:22.299467 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd0c37a-185c-4602-bcb8-69a17b79496c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:22.301600 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd0c37a-185c-4602-bcb8-69a17b79496c service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:22.305621 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102bfd8-4903-4243-a70a-c45e846f192e admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:22.310427 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f942d6-b481-415b-b511-9626f7116421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:22.311026 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f942d6-b481-415b-b511-9626f7116421 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:22.311303 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f942d6-b481-415b-b511-9626f7116421 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:22.311580 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f942d6-b481-415b-b511-9626f7116421 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:22.312255 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f942d6-b481-415b-b511-9626f7116421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:22.313233 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1301/2603] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:55:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 23:55:22.313846 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f942d6-b481-415b-b511-9626f7116421 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:22.326537 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1303/2604] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 23:55:22.330825 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd0c37a-185c-4602-bcb8-69a17b79496c service cinder] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:22.342187 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd0c37a-185c-4602-bcb8-69a17b79496c service cinder] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:22.348256 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae2973d-8596-4b5a-8c73-68f8da478345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:22.349021 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2973d-8596-4b5a-8c73-68f8da478345 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:22.349335 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2973d-8596-4b5a-8c73-68f8da478345 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:22.349585 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2973d-8596-4b5a-8c73-68f8da478345 admin admin] PATH_INFO: `/v3/projects/3e9796a76d0846aebbad8db721bbca45/users/49de8489be8048a0b52167d12ab05f8b/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:22.366001 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae2973d-8596-4b5a-8c73-68f8da478345 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e9796a76d0846aebbad8db721bbca45, user_id=49de8489be8048a0b52167d12ab05f8b, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:22.368703 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae2973d-8596-4b5a-8c73-68f8da478345 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:22.388018 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1302/2605] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:55:22 2022] PUT /identity/v3/projects/3e9796a76d0846aebbad8db721bbca45/users/49de8489be8048a0b52167d12ab05f8b/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:22.395239 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a9f08-d0d9-4c1c-b781-ced4334a3dba None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:22.395603 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a9f08-d0d9-4c1c-b781-ced4334a3dba None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:22.395947 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a9f08-d0d9-4c1c-b781-ced4334a3dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:22.431734 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-904a9f08-d0d9-4c1c-b781-ced4334a3dba None None] MFA Rules not processed for user `49de8489be8048a0b52167d12ab05f8b`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:22.433100 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-904a9f08-d0d9-4c1c-b781-ced4334a3dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:22.471334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:22.521769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1304/2606] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:55:22.529274 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae2973d-8596-4b5a-8c73-68f8da478345 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:22.531037 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-aae2973d-8596-4b5a-8c73-68f8da478345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:22.630650 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae2973d-8596-4b5a-8c73-68f8da478345 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:22.637292 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec57831-4933-4f07-9644-793db39231da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:22.637945 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec57831-4933-4f07-9644-793db39231da service nova] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:22.638227 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec57831-4933-4f07-9644-793db39231da service nova] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:22.638495 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec57831-4933-4f07-9644-793db39231da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:22.647809 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec57831-4933-4f07-9644-793db39231da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:22.649715 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec57831-4933-4f07-9644-793db39231da service nova] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:22.661219 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1303/2607] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:55:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:55:22.906371 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a9f08-d0d9-4c1c-b781-ced4334a3dba None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:22.991257 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a9f08-d0d9-4c1c-b781-ced4334a3dba None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:22.997034 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0d418-cab8-49f2-b592-c4324a3f3f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:22.998619 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0d418-cab8-49f2-b592-c4324a3f3f2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:22.999109 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0d418-cab8-49f2-b592-c4324a3f3f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:22.999812 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0d418-cab8-49f2-b592-c4324a3f3f2c admin admin] PATH_INFO: `/v3/users/49de8489be8048a0b52167d12ab05f8b` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:23.002903 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d0d418-cab8-49f2-b592-c4324a3f3f2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=49de8489be8048a0b52167d12ab05f8b)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:23.006767 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d0d418-cab8-49f2-b592-c4324a3f3f2c admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:23.088012 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-f0d0d418-cab8-49f2-b592-c4324a3f3f2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49de8489be8048a0b52167d12ab05f8b'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:23.089311 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-f0d0d418-cab8-49f2-b592-c4324a3f3f2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49de8489be8048a0b52167d12ab05f8b'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:23.090003 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-f0d0d418-cab8-49f2-b592-c4324a3f3f2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49de8489be8048a0b52167d12ab05f8b'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:23.103219 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-f0d0d418-cab8-49f2-b592-c4324a3f3f2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49de8489be8048a0b52167d12ab05f8b'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:23.122618 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-f0d0d418-cab8-49f2-b592-c4324a3f3f2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49de8489be8048a0b52167d12ab05f8b'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:23.161891 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1305/2608] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:55:22 2022] DELETE /identity/v3/users/49de8489be8048a0b52167d12ab05f8b => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:23.166461 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82666577-a1e4-4718-8396-3160463dc7fa None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:23.167220 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82666577-a1e4-4718-8396-3160463dc7fa None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:23.167309 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82666577-a1e4-4718-8396-3160463dc7fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:23.189692 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-82666577-a1e4-4718-8396-3160463dc7fa None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:23.191086 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-82666577-a1e4-4718-8396-3160463dc7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:23.304567 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1304/2609] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:55:23.324314 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0d418-cab8-49f2-b592-c4324a3f3f2c admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:23.326702 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-f0d0d418-cab8-49f2-b592-c4324a3f3f2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:23.406433 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0d418-cab8-49f2-b592-c4324a3f3f2c admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:23.410856 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959c47f-7b3e-4e09-b8b3-8641b642c4cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:23.412297 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959c47f-7b3e-4e09-b8b3-8641b642c4cf service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:23.412297 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959c47f-7b3e-4e09-b8b3-8641b642c4cf service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:23.412297 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959c47f-7b3e-4e09-b8b3-8641b642c4cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:23.424448 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:23.430768 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1959c47f-7b3e-4e09-b8b3-8641b642c4cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:23.432574 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1959c47f-7b3e-4e09-b8b3-8641b642c4cf service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:23.463295 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1306/2610] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:55:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:55:23.633415 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82666577-a1e4-4718-8396-3160463dc7fa None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:23.635385 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-82666577-a1e4-4718-8396-3160463dc7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:23.655510 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82666577-a1e4-4718-8396-3160463dc7fa None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:23.668710 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817797e0-b1aa-4969-8b0d-c2b24d686a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:23.669419 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817797e0-b1aa-4969-8b0d-c2b24d686a7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:23.669702 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817797e0-b1aa-4969-8b0d-c2b24d686a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:23.669983 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817797e0-b1aa-4969-8b0d-c2b24d686a7a admin admin] PATH_INFO: `/v3/users/64fdf46f185c48558ad1a637898745a9` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:23.671153 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817797e0-b1aa-4969-8b0d-c2b24d686a7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=64fdf46f185c48558ad1a637898745a9)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:23.673178 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817797e0-b1aa-4969-8b0d-c2b24d686a7a admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:23.727189 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:23.741923 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-817797e0-b1aa-4969-8b0d-c2b24d686a7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64fdf46f185c48558ad1a637898745a9'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:23.742752 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-817797e0-b1aa-4969-8b0d-c2b24d686a7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64fdf46f185c48558ad1a637898745a9'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:23.760654 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959c47f-7b3e-4e09-b8b3-8641b642c4cf service neutron] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:23.762134 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-817797e0-b1aa-4969-8b0d-c2b24d686a7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64fdf46f185c48558ad1a637898745a9'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:23.762259 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-1959c47f-7b3e-4e09-b8b3-8641b642c4cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:23.774315 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-817797e0-b1aa-4969-8b0d-c2b24d686a7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64fdf46f185c48558ad1a637898745a9'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:23.775184 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-817797e0-b1aa-4969-8b0d-c2b24d686a7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64fdf46f185c48558ad1a637898745a9'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:23.787396 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959c47f-7b3e-4e09-b8b3-8641b642c4cf service neutron] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:23.794978 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc6230-f6c8-4f15-9441-9092da55fbd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:23.795643 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc6230-f6c8-4f15-9441-9092da55fbd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:23.796057 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc6230-f6c8-4f15-9441-9092da55fbd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:23.796449 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc6230-f6c8-4f15-9441-9092da55fbd5 admin admin] PATH_INFO: `/v3/users/21e06d9bb5a240459ef3298a990a42f2` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:23.798175 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edc6230-f6c8-4f15-9441-9092da55fbd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21e06d9bb5a240459ef3298a990a42f2)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:23.800471 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edc6230-f6c8-4f15-9441-9092da55fbd5 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:23.815218 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1305/2611] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:55:23 2022] DELETE /identity/v3/users/64fdf46f185c48558ad1a637898745a9 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:23.874737 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-2edc6230-f6c8-4f15-9441-9092da55fbd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21e06d9bb5a240459ef3298a990a42f2'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:23.875749 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-2edc6230-f6c8-4f15-9441-9092da55fbd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21e06d9bb5a240459ef3298a990a42f2'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:23.876544 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-2edc6230-f6c8-4f15-9441-9092da55fbd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21e06d9bb5a240459ef3298a990a42f2'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:23.890602 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-2edc6230-f6c8-4f15-9441-9092da55fbd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21e06d9bb5a240459ef3298a990a42f2'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:23.899022 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-2edc6230-f6c8-4f15-9441-9092da55fbd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21e06d9bb5a240459ef3298a990a42f2'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:23.920732 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1307/2612] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:55:23 2022] DELETE /identity/v3/users/21e06d9bb5a240459ef3298a990a42f2 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:55:24.034443 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817797e0-b1aa-4969-8b0d-c2b24d686a7a admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:24.038984 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-817797e0-b1aa-4969-8b0d-c2b24d686a7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:24.090593 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc6230-f6c8-4f15-9441-9092da55fbd5 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:24.093210 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-2edc6230-f6c8-4f15-9441-9092da55fbd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:24.151365 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817797e0-b1aa-4969-8b0d-c2b24d686a7a admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:24.184050 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc6230-f6c8-4f15-9441-9092da55fbd5 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:24.189326 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b799d-0bce-46a8-a367-f3cd2f34e1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:24.190315 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b799d-0bce-46a8-a367-f3cd2f34e1e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:24.190638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b799d-0bce-46a8-a367-f3cd2f34e1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:24.191134 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b799d-0bce-46a8-a367-f3cd2f34e1e2 admin admin] PATH_INFO: `/v3/projects/190eae26a0d5487882d1c59c337f0db4` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:24.192410 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815b799d-0bce-46a8-a367-f3cd2f34e1e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=190eae26a0d5487882d1c59c337f0db4)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:24.194385 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815b799d-0bce-46a8-a367-f3cd2f34e1e2 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:24.220420 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf0cc15-a99a-4cdd-9449-a31843137568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:24.221046 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0cc15-a99a-4cdd-9449-a31843137568 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:24.221342 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0cc15-a99a-4cdd-9449-a31843137568 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:24.221610 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0cc15-a99a-4cdd-9449-a31843137568 admin admin] PATH_INFO: `/v3/projects/3e9796a76d0846aebbad8db721bbca45` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:24.223270 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf0cc15-a99a-4cdd-9449-a31843137568 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e9796a76d0846aebbad8db721bbca45)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:24.225181 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf0cc15-a99a-4cdd-9449-a31843137568 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:24.242799 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-815b799d-0bce-46a8-a367-f3cd2f34e1e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '190eae26a0d5487882d1c59c337f0db4', 'request_id': 'req-815b799d-0bce-46a8-a367-f3cd2f34e1e2'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:24.248803 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-815b799d-0bce-46a8-a367-f3cd2f34e1e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '190eae26a0d5487882d1c59c337f0db4', 'request_id': 'req-815b799d-0bce-46a8-a367-f3cd2f34e1e2'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:24.256076 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-815b799d-0bce-46a8-a367-f3cd2f34e1e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '190eae26a0d5487882d1c59c337f0db4', 'request_id': 'req-815b799d-0bce-46a8-a367-f3cd2f34e1e2'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:24.258135 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:24.270792 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-815b799d-0bce-46a8-a367-f3cd2f34e1e2 admin admin] The token cache is being invalidate because project 190eae26a0d5487882d1c59c337f0db4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:55:24.271015 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-815b799d-0bce-46a8-a367-f3cd2f34e1e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:24.271657 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-815b799d-0bce-46a8-a367-f3cd2f34e1e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:24.273167 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1308/2613] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:55:24 2022] DELETE /identity/v3/projects/190eae26a0d5487882d1c59c337f0db4 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:55:24.274901 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-daf0cc15-a99a-4cdd-9449-a31843137568 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e9796a76d0846aebbad8db721bbca45', 'request_id': 'req-daf0cc15-a99a-4cdd-9449-a31843137568'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:24.280733 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-daf0cc15-a99a-4cdd-9449-a31843137568 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e9796a76d0846aebbad8db721bbca45', 'request_id': 'req-daf0cc15-a99a-4cdd-9449-a31843137568'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:24.280975 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b799d-0bce-46a8-a367-f3cd2f34e1e2 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:24.287872 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-815b799d-0bce-46a8-a367-f3cd2f34e1e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:24.290288 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-daf0cc15-a99a-4cdd-9449-a31843137568 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e9796a76d0846aebbad8db721bbca45', 'request_id': 'req-daf0cc15-a99a-4cdd-9449-a31843137568'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:24.293623 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:24.306079 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-daf0cc15-a99a-4cdd-9449-a31843137568 admin admin] The token cache is being invalidate because project 3e9796a76d0846aebbad8db721bbca45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:55:24.306789 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-daf0cc15-a99a-4cdd-9449-a31843137568 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:24.307819 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-daf0cc15-a99a-4cdd-9449-a31843137568 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:24.310464 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1306/2614] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:55:24 2022] DELETE /identity/v3/projects/3e9796a76d0846aebbad8db721bbca45 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:24.315421 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72a1b2-d246-4c47-bcc0-825a7352bd4e None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:24.316167 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72a1b2-d246-4c47-bcc0-825a7352bd4e None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:24.316167 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72a1b2-d246-4c47-bcc0-825a7352bd4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:24.338374 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-ab72a1b2-d246-4c47-bcc0-825a7352bd4e None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:24.340122 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-ab72a1b2-d246-4c47-bcc0-825a7352bd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:24.371020 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b799d-0bce-46a8-a367-f3cd2f34e1e2 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:24.410628 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1307/2615] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:55:24.419324 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab45b9-444a-492c-8e53-95c76bbdad6a None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:24.419650 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab45b9-444a-492c-8e53-95c76bbdad6a None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:24.419913 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab45b9-444a-492c-8e53-95c76bbdad6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:24.433669 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f40e67b-e48e-4976-a1ce-969aefb55a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:24.434327 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f40e67b-e48e-4976-a1ce-969aefb55a94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:24.434605 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f40e67b-e48e-4976-a1ce-969aefb55a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:24.434877 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f40e67b-e48e-4976-a1ce-969aefb55a94 admin admin] PATH_INFO: `/v3/projects/258851d93c2d466a85150116d68441b6` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:24.436157 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f40e67b-e48e-4976-a1ce-969aefb55a94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=258851d93c2d466a85150116d68441b6)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:24.438336 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f40e67b-e48e-4976-a1ce-969aefb55a94 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:24.447182 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-5eab45b9-444a-492c-8e53-95c76bbdad6a None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:24.448947 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-5eab45b9-444a-492c-8e53-95c76bbdad6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:24.465313 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1308/2616] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:55:24.466499 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab45b9-444a-492c-8e53-95c76bbdad6a None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:24.479303 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab45b9-444a-492c-8e53-95c76bbdad6a None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:24.484283 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548bc5fe-28ee-48f1-be33-20ee7c6eb7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:24.485081 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548bc5fe-28ee-48f1-be33-20ee7c6eb7c4 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:24.485407 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548bc5fe-28ee-48f1-be33-20ee7c6eb7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:24.485690 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548bc5fe-28ee-48f1-be33-20ee7c6eb7c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:24.486348 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548bc5fe-28ee-48f1-be33-20ee7c6eb7c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:55:24.486654 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548bc5fe-28ee-48f1-be33-20ee7c6eb7c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:24.488368 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548bc5fe-28ee-48f1-be33-20ee7c6eb7c4 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:24.498128 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-7f40e67b-e48e-4976-a1ce-969aefb55a94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '258851d93c2d466a85150116d68441b6', 'request_id': 'req-7f40e67b-e48e-4976-a1ce-969aefb55a94'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:24.504556 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1309/2617] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:55:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:55:24.506123 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548bc5fe-28ee-48f1-be33-20ee7c6eb7c4 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:24.509466 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-7f40e67b-e48e-4976-a1ce-969aefb55a94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '258851d93c2d466a85150116d68441b6', 'request_id': 'req-7f40e67b-e48e-4976-a1ce-969aefb55a94'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:24.517315 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548bc5fe-28ee-48f1-be33-20ee7c6eb7c4 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:24.519612 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-7f40e67b-e48e-4976-a1ce-969aefb55a94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '258851d93c2d466a85150116d68441b6', 'request_id': 'req-7f40e67b-e48e-4976-a1ce-969aefb55a94'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:24.535120 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-7f40e67b-e48e-4976-a1ce-969aefb55a94 admin admin] The token cache is being invalidate because project 258851d93c2d466a85150116d68441b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:55:24.535343 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-7f40e67b-e48e-4976-a1ce-969aefb55a94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:24.536319 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-7f40e67b-e48e-4976-a1ce-969aefb55a94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:24.538224 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1309/2618] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:55:24 2022] DELETE /identity/v3/projects/258851d93c2d466a85150116d68441b6 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:24.543311 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9052f7d-3769-4d14-a08c-85fe876a8fe6 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:24.543598 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9052f7d-3769-4d14-a08c-85fe876a8fe6 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:24.543901 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9052f7d-3769-4d14-a08c-85fe876a8fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:24.564248 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-f9052f7d-3769-4d14-a08c-85fe876a8fe6 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:24.565457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-f9052f7d-3769-4d14-a08c-85fe876a8fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:24.577439 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad391d-b31e-4f47-af44-c30214904850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:24.578036 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad391d-b31e-4f47-af44-c30214904850 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:24.578305 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad391d-b31e-4f47-af44-c30214904850 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:24.578570 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad391d-b31e-4f47-af44-c30214904850 admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:24.579174 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ad391d-b31e-4f47-af44-c30214904850 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:24.581122 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ad391d-b31e-4f47-af44-c30214904850 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:24.594436 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:24.611371 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1310/2619] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:55:24 2022] POST /identity/v3/projects => generated 379 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:55:24.617101 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad391d-b31e-4f47-af44-c30214904850 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:24.619184 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-01ad391d-b31e-4f47-af44-c30214904850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:24.634031 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1310/2620] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:55:24.641813 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f72e5-abf9-41e8-8803-2f0768a83d98 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:24.642165 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f72e5-abf9-41e8-8803-2f0768a83d98 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:24.642586 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f72e5-abf9-41e8-8803-2f0768a83d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:24.663949 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-f31f72e5-abf9-41e8-8803-2f0768a83d98 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:24.665260 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-f31f72e5-abf9-41e8-8803-2f0768a83d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:24.687732 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad391d-b31e-4f47-af44-c30214904850 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:24.692918 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2edd7-912f-47fe-8c4b-576a91aa7452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:24.693782 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2edd7-912f-47fe-8c4b-576a91aa7452 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:24.694088 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2edd7-912f-47fe-8c4b-576a91aa7452 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:24.694323 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2edd7-912f-47fe-8c4b-576a91aa7452 admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:24.694957 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b2edd7-912f-47fe-8c4b-576a91aa7452 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:24.696905 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b2edd7-912f-47fe-8c4b-576a91aa7452 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:24.705573 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:24.719092 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1311/2621] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:55:24 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:55:24.723674 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2edd7-912f-47fe-8c4b-576a91aa7452 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:24.727014 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1311/2622] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:55:24.730934 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f72e5-abf9-41e8-8803-2f0768a83d98 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:24.733410 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2edd7-912f-47fe-8c4b-576a91aa7452 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:24.738632 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b61ae-e1e9-4b97-b6bb-be9ff5cf70ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:24.739216 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b61ae-e1e9-4b97-b6bb-be9ff5cf70ba admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:24.739492 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b61ae-e1e9-4b97-b6bb-be9ff5cf70ba admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:24.739757 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b61ae-e1e9-4b97-b6bb-be9ff5cf70ba admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:24.740244 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f72e5-abf9-41e8-8803-2f0768a83d98 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:24.740449 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820b61ae-e1e9-4b97-b6bb-be9ff5cf70ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:24.742027 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820b61ae-e1e9-4b97-b6bb-be9ff5cf70ba admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:24.744783 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a4f691-f11e-4b0a-8463-c8aaad49b02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:24.745358 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4f691-f11e-4b0a-8463-c8aaad49b02a admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:24.745635 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4f691-f11e-4b0a-8463-c8aaad49b02a admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:24.745908 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4f691-f11e-4b0a-8463-c8aaad49b02a admin admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:24.746546 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a4f691-f11e-4b0a-8463-c8aaad49b02a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:55:24.746846 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a4f691-f11e-4b0a-8463-c8aaad49b02a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:24.748436 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a4f691-f11e-4b0a-8463-c8aaad49b02a admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:24.756041 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1312/2623] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:55:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:55:24.757449 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b61ae-e1e9-4b97-b6bb-be9ff5cf70ba admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:24.762943 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1312/2624] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:55:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:55:24.764391 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a4f691-f11e-4b0a-8463-c8aaad49b02a admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:24.767281 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b61ae-e1e9-4b97-b6bb-be9ff5cf70ba admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:24.771903 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a2c5f9-98b6-46ec-a7f9-b405bf018809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:24.772498 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a2c5f9-98b6-46ec-a7f9-b405bf018809 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:24.772773 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a2c5f9-98b6-46ec-a7f9-b405bf018809 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:24.773112 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a2c5f9-98b6-46ec-a7f9-b405bf018809 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:24.773741 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a2c5f9-98b6-46ec-a7f9-b405bf018809 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:24.773856 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a4f691-f11e-4b0a-8463-c8aaad49b02a admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:24.775297 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a2c5f9-98b6-46ec-a7f9-b405bf018809 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:24.778286 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba70f15-83e4-426c-a8b8-ca57d2fb80b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:24.778854 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba70f15-83e4-426c-a8b8-ca57d2fb80b1 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:24.779130 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba70f15-83e4-426c-a8b8-ca57d2fb80b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:24.779408 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba70f15-83e4-426c-a8b8-ca57d2fb80b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:24.780032 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba70f15-83e4-426c-a8b8-ca57d2fb80b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:24.781899 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba70f15-83e4-426c-a8b8-ca57d2fb80b1 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:24.787566 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1313/2625] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:55:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:55:24.790092 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a2c5f9-98b6-46ec-a7f9-b405bf018809 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:24.801371 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a2c5f9-98b6-46ec-a7f9-b405bf018809 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:24.806550 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aecc2c-9e83-4bd7-a076-f71b9ec062e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:24.807312 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aecc2c-9e83-4bd7-a076-f71b9ec062e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:24.807592 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aecc2c-9e83-4bd7-a076-f71b9ec062e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:24.807867 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aecc2c-9e83-4bd7-a076-f71b9ec062e6 admin admin] PATH_INFO: `/v3/projects/1f1931852cff41cc81c43f4f7ec2a81c/users/9aac752590cf47b3a6e5ab5aad1f61b3/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:24.811999 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1313/2626] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:55:24 2022] POST /identity/v3/projects => generated 357 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:55:24.815560 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba70f15-83e4-426c-a8b8-ca57d2fb80b1 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:24.825560 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aecc2c-9e83-4bd7-a076-f71b9ec062e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f1931852cff41cc81c43f4f7ec2a81c, user_id=9aac752590cf47b3a6e5ab5aad1f61b3, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:24.828260 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aecc2c-9e83-4bd7-a076-f71b9ec062e6 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:24.841411 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1314/2627] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:55:24 2022] PUT /identity/v3/projects/1f1931852cff41cc81c43f4f7ec2a81c/users/9aac752590cf47b3a6e5ab5aad1f61b3/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:55:24.853294 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6d07a-63b4-4d55-a86b-c08d49976955 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:24.853576 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6d07a-63b4-4d55-a86b-c08d49976955 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:24.853851 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6d07a-63b4-4d55-a86b-c08d49976955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:24.883545 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-65d6d07a-63b4-4d55-a86b-c08d49976955 None None] MFA Rules not processed for user `9aac752590cf47b3a6e5ab5aad1f61b3`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:24.884121 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba70f15-83e4-426c-a8b8-ca57d2fb80b1 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:24.884777 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-65d6d07a-63b4-4d55-a86b-c08d49976955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:24.929569 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:24.944620 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c5ea05-8590-4a93-9b14-f7159d40d528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:24.945231 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5ea05-8590-4a93-9b14-f7159d40d528 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:24.945512 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5ea05-8590-4a93-9b14-f7159d40d528 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:24.945785 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5ea05-8590-4a93-9b14-f7159d40d528 admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:24.946413 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c5ea05-8590-4a93-9b14-f7159d40d528 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:24.948359 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c5ea05-8590-4a93-9b14-f7159d40d528 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:24.949978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1315/2628] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:55:24.954348 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d6d07a-63b4-4d55-a86b-c08d49976955 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:24.964580 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d6d07a-63b4-4d55-a86b-c08d49976955 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:24.969153 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f22cb-df42-4d4e-a7ba-91cadd0d83e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:24.969741 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f22cb-df42-4d4e-a7ba-91cadd0d83e4 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:24.970011 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f22cb-df42-4d4e-a7ba-91cadd0d83e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:24.970277 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f22cb-df42-4d4e-a7ba-91cadd0d83e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:24.970592 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1314/2629] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:55:24 2022] POST /identity/v3/users => generated 279 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:55:24.970898 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558f22cb-df42-4d4e-a7ba-91cadd0d83e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:24.973369 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558f22cb-df42-4d4e-a7ba-91cadd0d83e4 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:24.974803 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c5ea05-8590-4a93-9b14-f7159d40d528 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:24.984764 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c5ea05-8590-4a93-9b14-f7159d40d528 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:24.989926 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a42a93c-23a2-4d0d-a7fb-b6acfb8ff6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:24.990515 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a42a93c-23a2-4d0d-a7fb-b6acfb8ff6d3 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:24.990805 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a42a93c-23a2-4d0d-a7fb-b6acfb8ff6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:24.991083 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a42a93c-23a2-4d0d-a7fb-b6acfb8ff6d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:24.991875 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a42a93c-23a2-4d0d-a7fb-b6acfb8ff6d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:24.993933 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a42a93c-23a2-4d0d-a7fb-b6acfb8ff6d3 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:25.005982 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1316/2630] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:55:24 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:55:25.006136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1315/2631] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:55:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 23:55:25.009353 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a42a93c-23a2-4d0d-a7fb-b6acfb8ff6d3 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:25.009464 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f22cb-df42-4d4e-a7ba-91cadd0d83e4 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:25.020693 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a42a93c-23a2-4d0d-a7fb-b6acfb8ff6d3 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:25.076294 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f22cb-df42-4d4e-a7ba-91cadd0d83e4 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:25.084159 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4695606-a17c-46bd-8233-8d7f433e7a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:25.084778 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4695606-a17c-46bd-8233-8d7f433e7a53 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:25.085060 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4695606-a17c-46bd-8233-8d7f433e7a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:25.085334 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4695606-a17c-46bd-8233-8d7f433e7a53 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:25.085983 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4695606-a17c-46bd-8233-8d7f433e7a53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:25.087583 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4695606-a17c-46bd-8233-8d7f433e7a53 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:25.088677 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f538c4e-c49f-4c40-a1cc-4218f68d1991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:25.089283 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f538c4e-c49f-4c40-a1cc-4218f68d1991 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:25.089560 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f538c4e-c49f-4c40-a1cc-4218f68d1991 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:25.089830 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f538c4e-c49f-4c40-a1cc-4218f68d1991 admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:25.090448 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f538c4e-c49f-4c40-a1cc-4218f68d1991 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:25.092429 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f538c4e-c49f-4c40-a1cc-4218f68d1991 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:25.100112 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1317/2632] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 23:55:25.102655 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4695606-a17c-46bd-8233-8d7f433e7a53 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:25.116148 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4695606-a17c-46bd-8233-8d7f433e7a53 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:25.118569 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1316/2633] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:55:25 2022] POST /identity/v3/users => generated 297 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:55:25.120808 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff778380-9407-4abf-b4e3-18667fd23938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:25.121693 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff778380-9407-4abf-b4e3-18667fd23938 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:25.122148 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff778380-9407-4abf-b4e3-18667fd23938 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:25.122420 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff778380-9407-4abf-b4e3-18667fd23938 admin admin] PATH_INFO: `/v3/projects/4f72ce0661bf4104b3e4e53ed499813f/users/4a93c76294e84aeba4fd8a0fa96b0774/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:25.122526 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f538c4e-c49f-4c40-a1cc-4218f68d1991 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:25.126030 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:25.135620 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f538c4e-c49f-4c40-a1cc-4218f68d1991 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:25.140496 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4deb908-e6e2-46fa-a48d-758ace536025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:25.141207 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4deb908-e6e2-46fa-a48d-758ace536025 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:25.141620 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4deb908-e6e2-46fa-a48d-758ace536025 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:25.141971 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4deb908-e6e2-46fa-a48d-758ace536025 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:25.142689 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4deb908-e6e2-46fa-a48d-758ace536025 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:25.144458 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff778380-9407-4abf-b4e3-18667fd23938 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f72ce0661bf4104b3e4e53ed499813f, user_id=4a93c76294e84aeba4fd8a0fa96b0774, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:25.145453 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4deb908-e6e2-46fa-a48d-758ace536025 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:25.147072 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff778380-9407-4abf-b4e3-18667fd23938 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:25.156505 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1317/2634] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:55:25.160744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1318/2635] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:55:25 2022] PUT /identity/v3/projects/4f72ce0661bf4104b3e4e53ed499813f/users/4a93c76294e84aeba4fd8a0fa96b0774/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:55:25.161057 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4deb908-e6e2-46fa-a48d-758ace536025 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:25.171265 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4deb908-e6e2-46fa-a48d-758ace536025 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:25.172010 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1f4b0-1de4-42e6-83e7-b4f398e54a23 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:25.172299 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1f4b0-1de4-42e6-83e7-b4f398e54a23 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:25.172573 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1f4b0-1de4-42e6-83e7-b4f398e54a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:25.199689 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-62c1f4b0-1de4-42e6-83e7-b4f398e54a23 None None] MFA Rules not processed for user `4a93c76294e84aeba4fd8a0fa96b0774`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:25.200939 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-62c1f4b0-1de4-42e6-83e7-b4f398e54a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:25.229460 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d762e184-1f60-4826-a787-dfdc950b6271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:25.230143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762e184-1f60-4826-a787-dfdc950b6271 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:25.230427 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762e184-1f60-4826-a787-dfdc950b6271 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:25.230692 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762e184-1f60-4826-a787-dfdc950b6271 admin admin] PATH_INFO: `/v3/projects/5cb758ed0b3b4847841aa2ae3d3ae67c/users/401bc44b6bb246bfa3eaee525eae3aae/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:25.245037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d762e184-1f60-4826-a787-dfdc950b6271 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cb758ed0b3b4847841aa2ae3d3ae67c, user_id=401bc44b6bb246bfa3eaee525eae3aae, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:25.247785 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d762e184-1f60-4826-a787-dfdc950b6271 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:25.261498 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1318/2636] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:55:25 2022] PUT /identity/v3/projects/5cb758ed0b3b4847841aa2ae3d3ae67c/users/401bc44b6bb246bfa3eaee525eae3aae/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:55:25.265475 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d762e184-1f60-4826-a787-dfdc950b6271 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:25.270447 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1319/2637] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:25 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:55:25.274636 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1f4b0-1de4-42e6-83e7-b4f398e54a23 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:25.285187 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1f4b0-1de4-42e6-83e7-b4f398e54a23 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:25.298895 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:25.333549 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d762e184-1f60-4826-a787-dfdc950b6271 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:25.338139 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d65f55-ddbc-4eee-bdaa-cd650bbe3b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:25.338736 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d65f55-ddbc-4eee-bdaa-cd650bbe3b42 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:25.339013 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d65f55-ddbc-4eee-bdaa-cd650bbe3b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:25.339285 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d65f55-ddbc-4eee-bdaa-cd650bbe3b42 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:25.339923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d65f55-ddbc-4eee-bdaa-cd650bbe3b42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:25.341537 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d65f55-ddbc-4eee-bdaa-cd650bbe3b42 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:25.342701 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60667ed5-3706-4375-897a-3d82658f7b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:25.343303 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60667ed5-3706-4375-897a-3d82658f7b0a admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:25.343577 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60667ed5-3706-4375-897a-3d82658f7b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:25.343841 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60667ed5-3706-4375-897a-3d82658f7b0a admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:25.344479 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60667ed5-3706-4375-897a-3d82658f7b0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:25.346347 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60667ed5-3706-4375-897a-3d82658f7b0a admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:25.351911 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1319/2638] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 23:55:25.356022 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d65f55-ddbc-4eee-bdaa-cd650bbe3b42 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:25.365466 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d65f55-ddbc-4eee-bdaa-cd650bbe3b42 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:25.370465 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642dcb3-ce7e-47d0-a5c8-d40f4f863286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:25.371154 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642dcb3-ce7e-47d0-a5c8-d40f4f863286 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:25.371618 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642dcb3-ce7e-47d0-a5c8-d40f4f863286 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:25.371877 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642dcb3-ce7e-47d0-a5c8-d40f4f863286 admin admin] PATH_INFO: `/v3/projects/5cb758ed0b3b4847841aa2ae3d3ae67c/users/401bc44b6bb246bfa3eaee525eae3aae/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:25.373780 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d642dcb3-ce7e-47d0-a5c8-d40f4f863286 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cb758ed0b3b4847841aa2ae3d3ae67c, user_id=401bc44b6bb246bfa3eaee525eae3aae, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:25.374226 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1320/2639] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:55:25 2022] POST /identity/v3/projects => generated 353 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:55:25.378689 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60667ed5-3706-4375-897a-3d82658f7b0a admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:25.378982 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d642dcb3-ce7e-47d0-a5c8-d40f4f863286 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:25.389572 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1320/2640] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:55:25 2022] PUT /identity/v3/projects/5cb758ed0b3b4847841aa2ae3d3ae67c/users/401bc44b6bb246bfa3eaee525eae3aae/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:55:25.400193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea3b8c-158c-4e39-81bb-c7d452b4789b None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:25.400481 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea3b8c-158c-4e39-81bb-c7d452b4789b None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:25.400755 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea3b8c-158c-4e39-81bb-c7d452b4789b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:25.429038 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-abea3b8c-158c-4e39-81bb-c7d452b4789b None None] MFA Rules not processed for user `401bc44b6bb246bfa3eaee525eae3aae`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:25.430243 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-abea3b8c-158c-4e39-81bb-c7d452b4789b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:25.441860 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60667ed5-3706-4375-897a-3d82658f7b0a admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:25.445500 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:25.498182 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502ac18-c940-4361-92af-3c016383b564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:25.498784 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502ac18-c940-4361-92af-3c016383b564 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:25.499056 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502ac18-c940-4361-92af-3c016383b564 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:25.499321 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502ac18-c940-4361-92af-3c016383b564 admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:25.499929 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f502ac18-c940-4361-92af-3c016383b564 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:25.501864 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f502ac18-c940-4361-92af-3c016383b564 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:25.505896 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1321/2641] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:25 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:55:25.517815 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea3b8c-158c-4e39-81bb-c7d452b4789b None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:25.520111 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-abea3b8c-158c-4e39-81bb-c7d452b4789b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:25.524937 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1321/2642] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:55:25 2022] POST /identity/v3/users => generated 283 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:55:25.529228 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502ac18-c940-4361-92af-3c016383b564 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:25.540862 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502ac18-c940-4361-92af-3c016383b564 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:25.546284 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79fc2c-ac2a-412a-bd54-1c2973069a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:25.546877 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79fc2c-ac2a-412a-bd54-1c2973069a19 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:25.547156 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79fc2c-ac2a-412a-bd54-1c2973069a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:25.547427 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79fc2c-ac2a-412a-bd54-1c2973069a19 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:25.548115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b79fc2c-ac2a-412a-bd54-1c2973069a19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:25.549791 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b79fc2c-ac2a-412a-bd54-1c2973069a19 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:25.560598 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1322/2643] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:55:25.564778 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79fc2c-ac2a-412a-bd54-1c2973069a19 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:25.575351 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79fc2c-ac2a-412a-bd54-1c2973069a19 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:25.579944 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896e347-6147-48f9-8177-0e0861533b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:25.580635 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896e347-6147-48f9-8177-0e0861533b04 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:25.580936 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896e347-6147-48f9-8177-0e0861533b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:25.581228 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896e347-6147-48f9-8177-0e0861533b04 admin admin] PATH_INFO: `/v3/projects/524087aae8954053862e5a1c46b5bd1f/users/cbf2b089117841779810e2f9b57eeeaa/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:25.584237 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea3b8c-158c-4e39-81bb-c7d452b4789b None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:25.587772 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b79518-3811-4ff4-813b-e9f6cf944476 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:25.588421 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b79518-3811-4ff4-813b-e9f6cf944476 service cinder] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:25.588701 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b79518-3811-4ff4-813b-e9f6cf944476 service cinder] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:25.588973 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b79518-3811-4ff4-813b-e9f6cf944476 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:25.598129 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1896e347-6147-48f9-8177-0e0861533b04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=524087aae8954053862e5a1c46b5bd1f, user_id=cbf2b089117841779810e2f9b57eeeaa, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:25.599152 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b79518-3811-4ff4-813b-e9f6cf944476 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:25.600632 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b79518-3811-4ff4-813b-e9f6cf944476 service cinder] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:25.600821 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1896e347-6147-48f9-8177-0e0861533b04 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:25.610343 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1322/2644] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:55:25 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 23:55:25.613762 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1323/2645] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:55:25 2022] PUT /identity/v3/projects/524087aae8954053862e5a1c46b5bd1f/users/cbf2b089117841779810e2f9b57eeeaa/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:55:25.617724 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b79518-3811-4ff4-813b-e9f6cf944476 service cinder] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:25.628009 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b79518-3811-4ff4-813b-e9f6cf944476 service cinder] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:25.663937 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:25.686674 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896e347-6147-48f9-8177-0e0861533b04 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:25.695162 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34d97b1-495a-4380-84e4-52205a391706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:25.696241 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d97b1-495a-4380-84e4-52205a391706 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:25.696655 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d97b1-495a-4380-84e4-52205a391706 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:25.697457 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d97b1-495a-4380-84e4-52205a391706 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:25.697980 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34d97b1-495a-4380-84e4-52205a391706 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:25.700267 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34d97b1-495a-4380-84e4-52205a391706 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:25.711101 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1323/2646] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:55:25.715590 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34d97b1-495a-4380-84e4-52205a391706 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:25.725355 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34d97b1-495a-4380-84e4-52205a391706 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:25.730301 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023770e3-2650-4e04-805d-f045186ea0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:25.731275 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023770e3-2650-4e04-805d-f045186ea0b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:25.731558 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023770e3-2650-4e04-805d-f045186ea0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:25.731831 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023770e3-2650-4e04-805d-f045186ea0b3 admin admin] PATH_INFO: `/v3/projects/524087aae8954053862e5a1c46b5bd1f/users/cbf2b089117841779810e2f9b57eeeaa/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:25.734081 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023770e3-2650-4e04-805d-f045186ea0b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=524087aae8954053862e5a1c46b5bd1f, user_id=cbf2b089117841779810e2f9b57eeeaa, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:25.737248 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023770e3-2650-4e04-805d-f045186ea0b3 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:25.742978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896e347-6147-48f9-8177-0e0861533b04 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:25.747653 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1324/2647] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:55:25 2022] PUT /identity/v3/projects/524087aae8954053862e5a1c46b5bd1f/users/cbf2b089117841779810e2f9b57eeeaa/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:55:25.757736 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08aa936-5e76-4eb0-93fa-2eae25666cb3 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:25.758025 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08aa936-5e76-4eb0-93fa-2eae25666cb3 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:25.758298 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08aa936-5e76-4eb0-93fa-2eae25666cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:25.785940 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-a08aa936-5e76-4eb0-93fa-2eae25666cb3 None None] MFA Rules not processed for user `cbf2b089117841779810e2f9b57eeeaa`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:25.787188 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-a08aa936-5e76-4eb0-93fa-2eae25666cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:25.792380 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750258a9-6789-462e-8b81-f98eb21cfbcd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:25.793008 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750258a9-6789-462e-8b81-f98eb21cfbcd service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:25.793286 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750258a9-6789-462e-8b81-f98eb21cfbcd service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:25.793558 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750258a9-6789-462e-8b81-f98eb21cfbcd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:25.825082 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:25.850638 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750258a9-6789-462e-8b81-f98eb21cfbcd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:25.852096 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750258a9-6789-462e-8b81-f98eb21cfbcd service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:25.855917 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1325/2648] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:25 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:55:25.860864 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1324/2649] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:55:25 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 23:55:25.868057 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08aa936-5e76-4eb0-93fa-2eae25666cb3 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:25.869829 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-a08aa936-5e76-4eb0-93fa-2eae25666cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:25.883443 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08aa936-5e76-4eb0-93fa-2eae25666cb3 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:25.887095 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35205a-e377-4b4a-b3d4-f97a6c1aff6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:25.887696 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35205a-e377-4b4a-b3d4-f97a6c1aff6f service cinder] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:25.888000 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35205a-e377-4b4a-b3d4-f97a6c1aff6f service cinder] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:25.888282 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35205a-e377-4b4a-b3d4-f97a6c1aff6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:25.897374 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf35205a-e377-4b4a-b3d4-f97a6c1aff6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:25.898820 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf35205a-e377-4b4a-b3d4-f97a6c1aff6f service cinder] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:25.908300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1326/2650] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:55:25 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:55:25.986848 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750258a9-6789-462e-8b81-f98eb21cfbcd service cinder] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:25.996929 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750258a9-6789-462e-8b81-f98eb21cfbcd service cinder] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:26.000788 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c767118-3b25-455c-9ee8-adf008b21d18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:26.001425 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c767118-3b25-455c-9ee8-adf008b21d18 service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:26.001679 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c767118-3b25-455c-9ee8-adf008b21d18 service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:26.001943 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c767118-3b25-455c-9ee8-adf008b21d18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:26.061521 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c767118-3b25-455c-9ee8-adf008b21d18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:26.062948 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c767118-3b25-455c-9ee8-adf008b21d18 service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:26.072320 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1325/2651] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:55:25 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:55:35.904111 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=57,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:36.068334 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=41 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:41.881767 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35205a-e377-4b4a-b3d4-f97a6c1aff6f service cinder] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:41.942587 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35205a-e377-4b4a-b3d4-f97a6c1aff6f service cinder] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:41.947135 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc8676-231f-4d32-824a-d8cdff2c63cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:41.947752 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc8676-231f-4d32-824a-d8cdff2c63cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:41.948060 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc8676-231f-4d32-824a-d8cdff2c63cc admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:41.948333 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc8676-231f-4d32-824a-d8cdff2c63cc admin admin] PATH_INFO: `/v3/users/9aac752590cf47b3a6e5ab5aad1f61b3` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:41.949436 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc8676-231f-4d32-824a-d8cdff2c63cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9aac752590cf47b3a6e5ab5aad1f61b3)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:41.951344 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc8676-231f-4d32-824a-d8cdff2c63cc admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:41.995660 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-4ecc8676-231f-4d32-824a-d8cdff2c63cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9aac752590cf47b3a6e5ab5aad1f61b3'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:41.996268 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-4ecc8676-231f-4d32-824a-d8cdff2c63cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9aac752590cf47b3a6e5ab5aad1f61b3'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:41.996875 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-4ecc8676-231f-4d32-824a-d8cdff2c63cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9aac752590cf47b3a6e5ab5aad1f61b3'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:42.003996 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-4ecc8676-231f-4d32-824a-d8cdff2c63cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9aac752590cf47b3a6e5ab5aad1f61b3'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:42.011389 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-4ecc8676-231f-4d32-824a-d8cdff2c63cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9aac752590cf47b3a6e5ab5aad1f61b3'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:42.028474 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1327/2652] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:55:41 2022] DELETE /identity/v3/users/9aac752590cf47b3a6e5ab5aad1f61b3 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:42.032934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638cc8f9-6786-4b2b-a5e7-590187892977 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:42.033417 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638cc8f9-6786-4b2b-a5e7-590187892977 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:42.033528 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638cc8f9-6786-4b2b-a5e7-590187892977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:42.053796 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-638cc8f9-6786-4b2b-a5e7-590187892977 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:42.054994 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-638cc8f9-6786-4b2b-a5e7-590187892977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:42.115667 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1326/2653] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:55:42.126167 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc8676-231f-4d32-824a-d8cdff2c63cc admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:42.127861 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-4ecc8676-231f-4d32-824a-d8cdff2c63cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:42.185335 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc8676-231f-4d32-824a-d8cdff2c63cc admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:42.189030 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a34ff-305b-47e7-a437-874c6cf91006 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:42.189640 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a34ff-305b-47e7-a437-874c6cf91006 service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:42.189920 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a34ff-305b-47e7-a437-874c6cf91006 service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:42.190260 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a34ff-305b-47e7-a437-874c6cf91006 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:42.199343 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a34ff-305b-47e7-a437-874c6cf91006 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:42.200855 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a34ff-305b-47e7-a437-874c6cf91006 service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:42.211277 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1328/2654] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:55:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 23:55:42.507844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638cc8f9-6786-4b2b-a5e7-590187892977 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:42.509506 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-638cc8f9-6786-4b2b-a5e7-590187892977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:42.542572 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638cc8f9-6786-4b2b-a5e7-590187892977 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:42.557494 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57401e31-d13a-4bfb-a0b4-e9d6b0061420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:42.559023 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57401e31-d13a-4bfb-a0b4-e9d6b0061420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:42.559481 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57401e31-d13a-4bfb-a0b4-e9d6b0061420 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:42.559761 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57401e31-d13a-4bfb-a0b4-e9d6b0061420 admin admin] PATH_INFO: `/v3/projects/1f1931852cff41cc81c43f4f7ec2a81c` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:42.561351 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57401e31-d13a-4bfb-a0b4-e9d6b0061420 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f1931852cff41cc81c43f4f7ec2a81c)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:42.566179 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57401e31-d13a-4bfb-a0b4-e9d6b0061420 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:42.630162 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-57401e31-d13a-4bfb-a0b4-e9d6b0061420 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f1931852cff41cc81c43f4f7ec2a81c', 'request_id': 'req-57401e31-d13a-4bfb-a0b4-e9d6b0061420'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:42.636221 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-57401e31-d13a-4bfb-a0b4-e9d6b0061420 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f1931852cff41cc81c43f4f7ec2a81c', 'request_id': 'req-57401e31-d13a-4bfb-a0b4-e9d6b0061420'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:42.643453 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-57401e31-d13a-4bfb-a0b4-e9d6b0061420 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f1931852cff41cc81c43f4f7ec2a81c', 'request_id': 'req-57401e31-d13a-4bfb-a0b4-e9d6b0061420'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:42.655220 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-57401e31-d13a-4bfb-a0b4-e9d6b0061420 admin admin] The token cache is being invalidate because project 1f1931852cff41cc81c43f4f7ec2a81c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:55:42.655547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-57401e31-d13a-4bfb-a0b4-e9d6b0061420 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:42.656283 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-57401e31-d13a-4bfb-a0b4-e9d6b0061420 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:42.657805 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1327/2655] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:55:42 2022] DELETE /identity/v3/projects/1f1931852cff41cc81c43f4f7ec2a81c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:42.661951 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a34ff-305b-47e7-a437-874c6cf91006 service neutron] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:42.672353 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a34ff-305b-47e7-a437-874c6cf91006 service neutron] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:42.690245 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:42.729375 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cd66c8-abfc-461f-98d5-a13dbe862d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:42.730000 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd66c8-abfc-461f-98d5-a13dbe862d8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:42.730278 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd66c8-abfc-461f-98d5-a13dbe862d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:42.730550 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd66c8-abfc-461f-98d5-a13dbe862d8b admin admin] PATH_INFO: `/v3/users/401bc44b6bb246bfa3eaee525eae3aae` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:42.731681 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cd66c8-abfc-461f-98d5-a13dbe862d8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=401bc44b6bb246bfa3eaee525eae3aae)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:42.733650 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cd66c8-abfc-461f-98d5-a13dbe862d8b admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:42.778695 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-f6cd66c8-abfc-461f-98d5-a13dbe862d8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '401bc44b6bb246bfa3eaee525eae3aae'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:42.779281 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-f6cd66c8-abfc-461f-98d5-a13dbe862d8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '401bc44b6bb246bfa3eaee525eae3aae'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:42.779880 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-f6cd66c8-abfc-461f-98d5-a13dbe862d8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '401bc44b6bb246bfa3eaee525eae3aae'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:42.787057 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-f6cd66c8-abfc-461f-98d5-a13dbe862d8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '401bc44b6bb246bfa3eaee525eae3aae'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:42.794260 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-f6cd66c8-abfc-461f-98d5-a13dbe862d8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '401bc44b6bb246bfa3eaee525eae3aae'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:42.811056 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1329/2656] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:55:42 2022] DELETE /identity/v3/users/401bc44b6bb246bfa3eaee525eae3aae => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:43.075224 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57401e31-d13a-4bfb-a0b4-e9d6b0061420 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:43.076755 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-57401e31-d13a-4bfb-a0b4-e9d6b0061420 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:43.090373 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57401e31-d13a-4bfb-a0b4-e9d6b0061420 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:43.146447 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab6b33d-b1a3-4437-8bff-4c0a070a3717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:43.147070 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6b33d-b1a3-4437-8bff-4c0a070a3717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:43.147347 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6b33d-b1a3-4437-8bff-4c0a070a3717 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:43.147613 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6b33d-b1a3-4437-8bff-4c0a070a3717 admin admin] PATH_INFO: `/v3/projects/5cb758ed0b3b4847841aa2ae3d3ae67c` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:43.148752 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab6b33d-b1a3-4437-8bff-4c0a070a3717 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cb758ed0b3b4847841aa2ae3d3ae67c)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:43.150651 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab6b33d-b1a3-4437-8bff-4c0a070a3717 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:43.180043 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:43.197708 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-4ab6b33d-b1a3-4437-8bff-4c0a070a3717 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cb758ed0b3b4847841aa2ae3d3ae67c', 'request_id': 'req-4ab6b33d-b1a3-4437-8bff-4c0a070a3717'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:43.203138 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-4ab6b33d-b1a3-4437-8bff-4c0a070a3717 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cb758ed0b3b4847841aa2ae3d3ae67c', 'request_id': 'req-4ab6b33d-b1a3-4437-8bff-4c0a070a3717'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:43.209871 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-4ab6b33d-b1a3-4437-8bff-4c0a070a3717 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cb758ed0b3b4847841aa2ae3d3ae67c', 'request_id': 'req-4ab6b33d-b1a3-4437-8bff-4c0a070a3717'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:43.220877 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-4ab6b33d-b1a3-4437-8bff-4c0a070a3717 admin admin] The token cache is being invalidate because project 5cb758ed0b3b4847841aa2ae3d3ae67c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:55:43.221162 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-4ab6b33d-b1a3-4437-8bff-4c0a070a3717 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:43.221800 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-4ab6b33d-b1a3-4437-8bff-4c0a070a3717 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:43.223260 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1328/2657] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:55:43 2022] DELETE /identity/v3/projects/5cb758ed0b3b4847841aa2ae3d3ae67c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:43.228277 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0679c17-4d57-4b7e-9fa7-d1d655e67c54 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:43.228735 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0679c17-4d57-4b7e-9fa7-d1d655e67c54 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:43.228872 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0679c17-4d57-4b7e-9fa7-d1d655e67c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:43.248043 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-c0679c17-4d57-4b7e-9fa7-d1d655e67c54 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:43.249229 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-c0679c17-4d57-4b7e-9fa7-d1d655e67c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:43.307254 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1330/2658] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:55:43.316134 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b84eff-6f16-462b-8f1d-48048c8e78b7 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:43.317069 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b84eff-6f16-462b-8f1d-48048c8e78b7 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:43.317350 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b84eff-6f16-462b-8f1d-48048c8e78b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:43.336267 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-f6b84eff-6f16-462b-8f1d-48048c8e78b7 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:43.337473 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-f6b84eff-6f16-462b-8f1d-48048c8e78b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:43.345503 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1329/2659] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:55:43.350086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0679c17-4d57-4b7e-9fa7-d1d655e67c54 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:43.359655 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0679c17-4d57-4b7e-9fa7-d1d655e67c54 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:43.364823 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae7622-eb99-4373-9483-50b3c78fafae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:43.365414 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae7622-eb99-4373-9483-50b3c78fafae admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:43.365694 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae7622-eb99-4373-9483-50b3c78fafae admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:43.365967 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae7622-eb99-4373-9483-50b3c78fafae admin admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:43.366608 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae7622-eb99-4373-9483-50b3c78fafae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:55:43.366912 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae7622-eb99-4373-9483-50b3c78fafae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:43.368544 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae7622-eb99-4373-9483-50b3c78fafae admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:43.379652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1331/2660] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:55:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:55:43.383810 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b84eff-6f16-462b-8f1d-48048c8e78b7 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:43.392884 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b84eff-6f16-462b-8f1d-48048c8e78b7 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:43.397481 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b92567-3136-4d41-9601-1a4491d9db5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:43.398055 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b92567-3136-4d41-9601-1a4491d9db5b admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:43.398326 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b92567-3136-4d41-9601-1a4491d9db5b admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:43.398589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b92567-3136-4d41-9601-1a4491d9db5b admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:43.399190 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b92567-3136-4d41-9601-1a4491d9db5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:43.401179 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b92567-3136-4d41-9601-1a4491d9db5b admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:43.428029 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1330/2661] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:55:43 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:55:43.432167 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae7622-eb99-4373-9483-50b3c78fafae admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:43.442108 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae7622-eb99-4373-9483-50b3c78fafae admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:43.467715 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:43.500644 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f0260-41d6-439c-a2ac-10e3aa77af7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:43.501246 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f0260-41d6-439c-a2ac-10e3aa77af7a admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:43.501529 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f0260-41d6-439c-a2ac-10e3aa77af7a admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:43.501805 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f0260-41d6-439c-a2ac-10e3aa77af7a admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:43.502435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7f0260-41d6-439c-a2ac-10e3aa77af7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:43.504420 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7f0260-41d6-439c-a2ac-10e3aa77af7a admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:43.523691 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1332/2662] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:55:43 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:55:43.527829 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b92567-3136-4d41-9601-1a4491d9db5b admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:43.538369 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b92567-3136-4d41-9601-1a4491d9db5b admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:43.543061 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed5e03-8e66-4e09-8cb3-52537bdae671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:43.543642 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed5e03-8e66-4e09-8cb3-52537bdae671 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:43.543915 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed5e03-8e66-4e09-8cb3-52537bdae671 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:43.544213 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed5e03-8e66-4e09-8cb3-52537bdae671 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:43.544843 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ed5e03-8e66-4e09-8cb3-52537bdae671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:43.546420 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ed5e03-8e66-4e09-8cb3-52537bdae671 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:43.556616 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1331/2663] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:55:43.560371 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f0260-41d6-439c-a2ac-10e3aa77af7a admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:43.570296 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f0260-41d6-439c-a2ac-10e3aa77af7a admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:43.574937 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b56314c-d2d7-446e-a3b2-be73cdede37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:43.575532 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56314c-d2d7-446e-a3b2-be73cdede37d admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:43.575815 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56314c-d2d7-446e-a3b2-be73cdede37d admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:43.576126 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56314c-d2d7-446e-a3b2-be73cdede37d admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:43.576807 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b56314c-d2d7-446e-a3b2-be73cdede37d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:43.578391 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b56314c-d2d7-446e-a3b2-be73cdede37d admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:43.589059 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1333/2664] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:55:43.593032 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed5e03-8e66-4e09-8cb3-52537bdae671 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:43.603641 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed5e03-8e66-4e09-8cb3-52537bdae671 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:43.608336 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ad274-ce42-4f30-a0e5-06fea4c33b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:43.608973 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ad274-ce42-4f30-a0e5-06fea4c33b83 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:43.609247 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ad274-ce42-4f30-a0e5-06fea4c33b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:43.609513 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ad274-ce42-4f30-a0e5-06fea4c33b83 admin admin] PATH_INFO: `/v3/projects/13748410a47342c18735294b2c97dcab/users/f39477414d9f4a1fa37921815e68bf10/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:43.624148 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2ad274-ce42-4f30-a0e5-06fea4c33b83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13748410a47342c18735294b2c97dcab, user_id=f39477414d9f4a1fa37921815e68bf10, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:43.626896 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2ad274-ce42-4f30-a0e5-06fea4c33b83 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:43.640035 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1332/2665] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:55:43 2022] PUT /identity/v3/projects/13748410a47342c18735294b2c97dcab/users/f39477414d9f4a1fa37921815e68bf10/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:43.650652 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0474d-944f-488d-ba35-ba541cec964b None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:43.650988 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0474d-944f-488d-ba35-ba541cec964b None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:43.651267 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0474d-944f-488d-ba35-ba541cec964b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:43.682531 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-76f0474d-944f-488d-ba35-ba541cec964b None None] MFA Rules not processed for user `f39477414d9f4a1fa37921815e68bf10`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:43.683763 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-76f0474d-944f-488d-ba35-ba541cec964b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:43.751948 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1334/2666] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:43 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:55:43.756468 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ad274-ce42-4f30-a0e5-06fea4c33b83 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:43.821253 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ad274-ce42-4f30-a0e5-06fea4c33b83 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:43.826446 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ea996-f44e-440a-84f0-72d92a054e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:43.827073 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ea996-f44e-440a-84f0-72d92a054e1a admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:43.827350 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ea996-f44e-440a-84f0-72d92a054e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:43.827617 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ea996-f44e-440a-84f0-72d92a054e1a admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:43.828289 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212ea996-f44e-440a-84f0-72d92a054e1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:43.830164 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212ea996-f44e-440a-84f0-72d92a054e1a admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:43.860003 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1333/2667] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:55:43 2022] POST /identity/v3/projects => generated 393 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:55:43.863035 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f0474d-944f-488d-ba35-ba541cec964b None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:43.874989 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f0474d-944f-488d-ba35-ba541cec964b None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:43.944595 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d8b8dd-9d9a-42a5-9cf1-b5a8f1880338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:43.945306 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8b8dd-9d9a-42a5-9cf1-b5a8f1880338 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:43.945661 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8b8dd-9d9a-42a5-9cf1-b5a8f1880338 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:43.946010 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8b8dd-9d9a-42a5-9cf1-b5a8f1880338 admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:43.946716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d8b8dd-9d9a-42a5-9cf1-b5a8f1880338 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:43.948737 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d8b8dd-9d9a-42a5-9cf1-b5a8f1880338 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:43.979508 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1335/2668] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:55:43 2022] POST /identity/v3/users => generated 303 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:55:43.980955 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ea996-f44e-440a-84f0-72d92a054e1a admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:43.994114 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ea996-f44e-440a-84f0-72d92a054e1a admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:44.000355 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1d729-08ee-4955-9e5d-37105fd0f547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:44.001036 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1d729-08ee-4955-9e5d-37105fd0f547 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:44.001427 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1d729-08ee-4955-9e5d-37105fd0f547 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:44.001725 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1d729-08ee-4955-9e5d-37105fd0f547 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:44.002373 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1d729-08ee-4955-9e5d-37105fd0f547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:44.004143 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1d729-08ee-4955-9e5d-37105fd0f547 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:44.016176 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1334/2669] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:55:44.020838 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d8b8dd-9d9a-42a5-9cf1-b5a8f1880338 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:44.031668 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d8b8dd-9d9a-42a5-9cf1-b5a8f1880338 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:44.037201 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdefb258-8e2b-48b2-ac29-b6c3633d01e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:44.037864 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefb258-8e2b-48b2-ac29-b6c3633d01e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:44.038157 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefb258-8e2b-48b2-ac29-b6c3633d01e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:44.038427 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefb258-8e2b-48b2-ac29-b6c3633d01e1 admin admin] PATH_INFO: `/v3/projects/8f46bc305f8041c29d4e0888ac9aa905/users/6a70893f9eb248a0b1adfdd38fee265b/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:44.053301 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdefb258-8e2b-48b2-ac29-b6c3633d01e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f46bc305f8041c29d4e0888ac9aa905, user_id=6a70893f9eb248a0b1adfdd38fee265b, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:44.055978 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdefb258-8e2b-48b2-ac29-b6c3633d01e1 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:44.068047 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1336/2670] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:55:44 2022] PUT /identity/v3/projects/8f46bc305f8041c29d4e0888ac9aa905/users/6a70893f9eb248a0b1adfdd38fee265b/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:44.071953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1d729-08ee-4955-9e5d-37105fd0f547 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:44.081906 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1d729-08ee-4955-9e5d-37105fd0f547 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:44.102157 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:44.140331 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccf0a0b-fedd-4c25-8fd2-23fa612194a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:44.140930 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf0a0b-fedd-4c25-8fd2-23fa612194a6 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:44.141208 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf0a0b-fedd-4c25-8fd2-23fa612194a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:44.141475 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf0a0b-fedd-4c25-8fd2-23fa612194a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:44.142111 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccf0a0b-fedd-4c25-8fd2-23fa612194a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:44.143699 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccf0a0b-fedd-4c25-8fd2-23fa612194a6 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:44.153678 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1335/2671] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:55:44 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:55:44.157968 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdefb258-8e2b-48b2-ac29-b6c3633d01e1 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:44.163362 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:44.169863 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdefb258-8e2b-48b2-ac29-b6c3633d01e1 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:44.174693 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692c73f-52a1-4282-955e-597af3b128f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:44.175331 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692c73f-52a1-4282-955e-597af3b128f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:44.175607 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692c73f-52a1-4282-955e-597af3b128f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:44.175907 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692c73f-52a1-4282-955e-597af3b128f0 admin admin] PATH_INFO: `/v3/projects/8f46bc305f8041c29d4e0888ac9aa905/users/6a70893f9eb248a0b1adfdd38fee265b/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:44.177806 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b692c73f-52a1-4282-955e-597af3b128f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f46bc305f8041c29d4e0888ac9aa905, user_id=6a70893f9eb248a0b1adfdd38fee265b, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:44.180485 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b692c73f-52a1-4282-955e-597af3b128f0 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:44.190598 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1337/2672] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:55:44 2022] PUT /identity/v3/projects/8f46bc305f8041c29d4e0888ac9aa905/users/6a70893f9eb248a0b1adfdd38fee265b/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:44.201427 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1f5e3-6928-48df-bafb-c024604e0453 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:44.201710 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1f5e3-6928-48df-bafb-c024604e0453 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:44.201979 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1f5e3-6928-48df-bafb-c024604e0453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:44.229373 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-8ad1f5e3-6928-48df-bafb-c024604e0453 None None] MFA Rules not processed for user `6a70893f9eb248a0b1adfdd38fee265b`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:44.230564 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-8ad1f5e3-6928-48df-bafb-c024604e0453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:44.296118 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1336/2673] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:44 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:55:44.306895 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692c73f-52a1-4282-955e-597af3b128f0 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:44.308635 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-b692c73f-52a1-4282-955e-597af3b128f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:44.364466 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692c73f-52a1-4282-955e-597af3b128f0 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:44.368228 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24cd79-74e7-4cf4-bf65-d334d1e78bfa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:44.368816 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24cd79-74e7-4cf4-bf65-d334d1e78bfa service cinder] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:44.369093 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24cd79-74e7-4cf4-bf65-d334d1e78bfa service cinder] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:44.369362 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24cd79-74e7-4cf4-bf65-d334d1e78bfa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:44.378143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a24cd79-74e7-4cf4-bf65-d334d1e78bfa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:44.380077 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a24cd79-74e7-4cf4-bf65-d334d1e78bfa service cinder] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:44.389200 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1338/2674] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:55:44 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:55:44.438410 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad1f5e3-6928-48df-bafb-c024604e0453 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:44.439775 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-8ad1f5e3-6928-48df-bafb-c024604e0453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:44.452461 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad1f5e3-6928-48df-bafb-c024604e0453 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:44.456505 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64cf21-bee2-4832-900e-02371e5971c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:44.457652 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64cf21-bee2-4832-900e-02371e5971c0 service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:44.457934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64cf21-bee2-4832-900e-02371e5971c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:44.458201 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64cf21-bee2-4832-900e-02371e5971c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:44.513263 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e64cf21-bee2-4832-900e-02371e5971c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:44.514694 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e64cf21-bee2-4832-900e-02371e5971c0 service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:44.523716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1337/2675] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:55:44 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:55:50.969283 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24cd79-74e7-4cf4-bf65-d334d1e78bfa service cinder] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:51.031781 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24cd79-74e7-4cf4-bf65-d334d1e78bfa service cinder] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:51.036330 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2bcf3f-0809-4932-a702-265172bf8d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:51.036955 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2bcf3f-0809-4932-a702-265172bf8d54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:51.037773 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2bcf3f-0809-4932-a702-265172bf8d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:51.037773 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2bcf3f-0809-4932-a702-265172bf8d54 admin admin] PATH_INFO: `/v3/users/f39477414d9f4a1fa37921815e68bf10` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:51.038695 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2bcf3f-0809-4932-a702-265172bf8d54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f39477414d9f4a1fa37921815e68bf10)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:51.041118 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2bcf3f-0809-4932-a702-265172bf8d54 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:51.094114 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-be2bcf3f-0809-4932-a702-265172bf8d54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f39477414d9f4a1fa37921815e68bf10'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:51.094929 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-be2bcf3f-0809-4932-a702-265172bf8d54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f39477414d9f4a1fa37921815e68bf10'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:51.095646 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-be2bcf3f-0809-4932-a702-265172bf8d54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f39477414d9f4a1fa37921815e68bf10'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:51.103546 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-be2bcf3f-0809-4932-a702-265172bf8d54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f39477414d9f4a1fa37921815e68bf10'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:51.111184 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-be2bcf3f-0809-4932-a702-265172bf8d54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f39477414d9f4a1fa37921815e68bf10'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:51.128718 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1339/2676] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:55:50 2022] DELETE /identity/v3/users/f39477414d9f4a1fa37921815e68bf10 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:51.133473 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3829b53-2ead-4c9c-9234-f79d390f0e9c None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:51.133953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3829b53-2ead-4c9c-9234-f79d390f0e9c None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:51.134058 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3829b53-2ead-4c9c-9234-f79d390f0e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:51.152929 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-e3829b53-2ead-4c9c-9234-f79d390f0e9c None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:51.154120 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-e3829b53-2ead-4c9c-9234-f79d390f0e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:51.192817 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:51.211878 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1338/2677] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:55:51.221487 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2bcf3f-0809-4932-a702-265172bf8d54 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:51.222856 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-be2bcf3f-0809-4932-a702-265172bf8d54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:51.236829 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:51.282253 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2bcf3f-0809-4932-a702-265172bf8d54 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:51.285782 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaa7759-60db-465a-b275-a524417f2ecf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:51.286378 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa7759-60db-465a-b275-a524417f2ecf service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:51.286650 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa7759-60db-465a-b275-a524417f2ecf service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:51.286918 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa7759-60db-465a-b275-a524417f2ecf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:51.296140 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaa7759-60db-465a-b275-a524417f2ecf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:51.297563 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaa7759-60db-465a-b275-a524417f2ecf service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:51.307689 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1340/2678] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:55:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:55:51.599232 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3829b53-2ead-4c9c-9234-f79d390f0e9c None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:51.600714 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-e3829b53-2ead-4c9c-9234-f79d390f0e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:51.612782 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3829b53-2ead-4c9c-9234-f79d390f0e9c None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:51.619029 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52635c86-8553-4394-8db5-1fbb8bcaa6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:51.619860 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52635c86-8553-4394-8db5-1fbb8bcaa6f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:51.620171 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52635c86-8553-4394-8db5-1fbb8bcaa6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:51.620949 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52635c86-8553-4394-8db5-1fbb8bcaa6f1 admin admin] PATH_INFO: `/v3/projects/13748410a47342c18735294b2c97dcab` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:51.622088 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52635c86-8553-4394-8db5-1fbb8bcaa6f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13748410a47342c18735294b2c97dcab)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:51.624716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52635c86-8553-4394-8db5-1fbb8bcaa6f1 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:51.673145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-52635c86-8553-4394-8db5-1fbb8bcaa6f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13748410a47342c18735294b2c97dcab', 'request_id': 'req-52635c86-8553-4394-8db5-1fbb8bcaa6f1'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:51.678571 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-52635c86-8553-4394-8db5-1fbb8bcaa6f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13748410a47342c18735294b2c97dcab', 'request_id': 'req-52635c86-8553-4394-8db5-1fbb8bcaa6f1'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:51.687378 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-52635c86-8553-4394-8db5-1fbb8bcaa6f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13748410a47342c18735294b2c97dcab', 'request_id': 'req-52635c86-8553-4394-8db5-1fbb8bcaa6f1'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:51.698290 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-52635c86-8553-4394-8db5-1fbb8bcaa6f1 admin admin] The token cache is being invalidate because project 13748410a47342c18735294b2c97dcab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:55:51.698572 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-52635c86-8553-4394-8db5-1fbb8bcaa6f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:51.699203 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-52635c86-8553-4394-8db5-1fbb8bcaa6f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:51.700685 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1339/2679] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:55:51 2022] DELETE /identity/v3/projects/13748410a47342c18735294b2c97dcab => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:51.704683 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaa7759-60db-465a-b275-a524417f2ecf service neutron] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:51.714085 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaa7759-60db-465a-b275-a524417f2ecf service neutron] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:51.773035 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da925f3-e469-4881-b2df-e87b4ed7560e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:51.773667 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da925f3-e469-4881-b2df-e87b4ed7560e admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:51.773942 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da925f3-e469-4881-b2df-e87b4ed7560e admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:51.774210 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da925f3-e469-4881-b2df-e87b4ed7560e admin admin] PATH_INFO: `/v3/users/6a70893f9eb248a0b1adfdd38fee265b` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:51.775329 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da925f3-e469-4881-b2df-e87b4ed7560e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a70893f9eb248a0b1adfdd38fee265b)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:51.777472 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da925f3-e469-4881-b2df-e87b4ed7560e admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:51.831291 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-2da925f3-e469-4881-b2df-e87b4ed7560e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a70893f9eb248a0b1adfdd38fee265b'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:51.831943 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-2da925f3-e469-4881-b2df-e87b4ed7560e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a70893f9eb248a0b1adfdd38fee265b'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:51.832603 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-2da925f3-e469-4881-b2df-e87b4ed7560e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a70893f9eb248a0b1adfdd38fee265b'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:51.840339 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-2da925f3-e469-4881-b2df-e87b4ed7560e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a70893f9eb248a0b1adfdd38fee265b'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:51.847382 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-2da925f3-e469-4881-b2df-e87b4ed7560e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a70893f9eb248a0b1adfdd38fee265b'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:51.865019 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1341/2680] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:55:51 2022] DELETE /identity/v3/users/6a70893f9eb248a0b1adfdd38fee265b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:55:52.144573 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52635c86-8553-4394-8db5-1fbb8bcaa6f1 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:52.146675 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-52635c86-8553-4394-8db5-1fbb8bcaa6f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:52.162651 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52635c86-8553-4394-8db5-1fbb8bcaa6f1 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:52.222541 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc6802-2c5c-4c23-928d-b8cc83b283ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:52.223002 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc6802-2c5c-4c23-928d-b8cc83b283ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:52.223279 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc6802-2c5c-4c23-928d-b8cc83b283ef admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:52.223554 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc6802-2c5c-4c23-928d-b8cc83b283ef admin admin] PATH_INFO: `/v3/projects/8f46bc305f8041c29d4e0888ac9aa905` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:52.224807 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc6802-2c5c-4c23-928d-b8cc83b283ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f46bc305f8041c29d4e0888ac9aa905)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:52.226766 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc6802-2c5c-4c23-928d-b8cc83b283ef admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:52.273844 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-05fc6802-2c5c-4c23-928d-b8cc83b283ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f46bc305f8041c29d4e0888ac9aa905', 'request_id': 'req-05fc6802-2c5c-4c23-928d-b8cc83b283ef'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:52.279949 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-05fc6802-2c5c-4c23-928d-b8cc83b283ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f46bc305f8041c29d4e0888ac9aa905', 'request_id': 'req-05fc6802-2c5c-4c23-928d-b8cc83b283ef'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:52.287630 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-05fc6802-2c5c-4c23-928d-b8cc83b283ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f46bc305f8041c29d4e0888ac9aa905', 'request_id': 'req-05fc6802-2c5c-4c23-928d-b8cc83b283ef'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:52.295810 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:52.302548 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-05fc6802-2c5c-4c23-928d-b8cc83b283ef admin admin] The token cache is being invalidate because project 8f46bc305f8041c29d4e0888ac9aa905 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:55:52.302548 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-05fc6802-2c5c-4c23-928d-b8cc83b283ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:52.302548 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-05fc6802-2c5c-4c23-928d-b8cc83b283ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:52.303649 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1340/2681] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:55:52 2022] DELETE /identity/v3/projects/8f46bc305f8041c29d4e0888ac9aa905 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:52.309115 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f633353-b798-4126-81d9-0a46792badea None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:52.309604 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f633353-b798-4126-81d9-0a46792badea None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:52.309785 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f633353-b798-4126-81d9-0a46792badea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:52.330480 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-5f633353-b798-4126-81d9-0a46792badea None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:52.331732 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-5f633353-b798-4126-81d9-0a46792badea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:52.362835 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:52.398291 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1342/2682] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:55:52.407600 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06f861-1eda-4dde-9dfd-a67b1d0b802e None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:52.407887 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06f861-1eda-4dde-9dfd-a67b1d0b802e None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:52.408199 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06f861-1eda-4dde-9dfd-a67b1d0b802e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:52.427663 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-9a06f861-1eda-4dde-9dfd-a67b1d0b802e None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:52.428899 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-9a06f861-1eda-4dde-9dfd-a67b1d0b802e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:52.436855 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1341/2683] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:55:52.445113 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f633353-b798-4126-81d9-0a46792badea None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:52.459558 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f633353-b798-4126-81d9-0a46792badea None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:52.466559 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57082f4d-7619-47e3-8868-6fc92ce74241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:52.467167 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57082f4d-7619-47e3-8868-6fc92ce74241 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:52.467450 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57082f4d-7619-47e3-8868-6fc92ce74241 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:52.467723 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57082f4d-7619-47e3-8868-6fc92ce74241 admin admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:52.468414 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57082f4d-7619-47e3-8868-6fc92ce74241 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:55:52.468721 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57082f4d-7619-47e3-8868-6fc92ce74241 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:52.470338 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57082f4d-7619-47e3-8868-6fc92ce74241 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:52.482985 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1343/2684] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:55:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:55:52.488690 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a06f861-1eda-4dde-9dfd-a67b1d0b802e None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:52.498503 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a06f861-1eda-4dde-9dfd-a67b1d0b802e None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:52.503141 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349c8cd-fd6b-4251-b2d7-07d958e0fc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:52.503747 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349c8cd-fd6b-4251-b2d7-07d958e0fc8b admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:52.504047 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349c8cd-fd6b-4251-b2d7-07d958e0fc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:52.504325 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349c8cd-fd6b-4251-b2d7-07d958e0fc8b admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:52.505019 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0349c8cd-fd6b-4251-b2d7-07d958e0fc8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:52.507015 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0349c8cd-fd6b-4251-b2d7-07d958e0fc8b admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:52.542022 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1342/2685] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:55:52 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:55:52.546729 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57082f4d-7619-47e3-8868-6fc92ce74241 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:52.560991 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57082f4d-7619-47e3-8868-6fc92ce74241 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:52.623867 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff3efd1-e57b-4d22-a2f9-9ce089bcde5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:52.624642 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3efd1-e57b-4d22-a2f9-9ce089bcde5e admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:52.624776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3efd1-e57b-4d22-a2f9-9ce089bcde5e admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:52.625288 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3efd1-e57b-4d22-a2f9-9ce089bcde5e admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:52.626053 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff3efd1-e57b-4d22-a2f9-9ce089bcde5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:52.628021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff3efd1-e57b-4d22-a2f9-9ce089bcde5e admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:52.650964 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1344/2686] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:55:52 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:55:52.655412 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349c8cd-fd6b-4251-b2d7-07d958e0fc8b admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:52.658558 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff3efd1-e57b-4d22-a2f9-9ce089bcde5e admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:52.660291 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-8ff3efd1-e57b-4d22-a2f9-9ce089bcde5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:52.665833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349c8cd-fd6b-4251-b2d7-07d958e0fc8b admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:52.670694 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b960d50-dd6a-48f0-ba12-6d122c519210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:52.671290 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b960d50-dd6a-48f0-ba12-6d122c519210 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:52.671569 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b960d50-dd6a-48f0-ba12-6d122c519210 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:52.671838 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b960d50-dd6a-48f0-ba12-6d122c519210 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:52.672515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b960d50-dd6a-48f0-ba12-6d122c519210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:52.672629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff3efd1-e57b-4d22-a2f9-9ce089bcde5e admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:52.674110 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b960d50-dd6a-48f0-ba12-6d122c519210 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:52.677491 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f88c0-d483-4565-a322-d92256acf0fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:52.678226 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f88c0-d483-4565-a322-d92256acf0fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:52.678603 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f88c0-d483-4565-a322-d92256acf0fa admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:52.678968 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f88c0-d483-4565-a322-d92256acf0fa admin admin] PATH_INFO: `/v3/users/90fc6085547a42099a8e6dbf032290f5` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:52.681322 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1f88c0-d483-4565-a322-d92256acf0fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=90fc6085547a42099a8e6dbf032290f5)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:52.683243 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1f88c0-d483-4565-a322-d92256acf0fa admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:52.685048 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1343/2687] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:55:52.688509 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b960d50-dd6a-48f0-ba12-6d122c519210 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:52.698001 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b960d50-dd6a-48f0-ba12-6d122c519210 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:52.702587 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7f318-09cb-431a-8f2f-6f3ca74b8966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:52.703169 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7f318-09cb-431a-8f2f-6f3ca74b8966 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:52.703447 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7f318-09cb-431a-8f2f-6f3ca74b8966 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:52.703715 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7f318-09cb-431a-8f2f-6f3ca74b8966 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:52.704379 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec7f318-09cb-431a-8f2f-6f3ca74b8966 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:52.705932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec7f318-09cb-431a-8f2f-6f3ca74b8966 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:52.717642 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1344/2688] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 23:55:52.722043 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7f318-09cb-431a-8f2f-6f3ca74b8966 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:52.732976 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7f318-09cb-431a-8f2f-6f3ca74b8966 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:52.733118 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-4d1f88c0-d483-4565-a322-d92256acf0fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90fc6085547a42099a8e6dbf032290f5'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:52.733678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-4d1f88c0-d483-4565-a322-d92256acf0fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90fc6085547a42099a8e6dbf032290f5'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:52.734291 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-4d1f88c0-d483-4565-a322-d92256acf0fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90fc6085547a42099a8e6dbf032290f5'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:52.737809 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c343029e-68fd-493c-9e5d-1ebd258326b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:52.738444 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343029e-68fd-493c-9e5d-1ebd258326b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:52.738735 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343029e-68fd-493c-9e5d-1ebd258326b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:52.739085 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343029e-68fd-493c-9e5d-1ebd258326b4 admin admin] PATH_INFO: `/v3/projects/37b7602604ce4244b6bfc2d5a6d26733/users/4cf79da3a4194eb0919885454ff1f38f/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:52.743818 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-4d1f88c0-d483-4565-a322-d92256acf0fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90fc6085547a42099a8e6dbf032290f5'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:52.751463 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-4d1f88c0-d483-4565-a322-d92256acf0fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90fc6085547a42099a8e6dbf032290f5'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:52.754045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c343029e-68fd-493c-9e5d-1ebd258326b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37b7602604ce4244b6bfc2d5a6d26733, user_id=4cf79da3a4194eb0919885454ff1f38f, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:52.756911 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c343029e-68fd-493c-9e5d-1ebd258326b4 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:52.769910 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1345/2689] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:55:52 2022] DELETE /identity/v3/users/90fc6085547a42099a8e6dbf032290f5 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:55:52.770403 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1345/2690] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:55:52 2022] PUT /identity/v3/projects/37b7602604ce4244b6bfc2d5a6d26733/users/4cf79da3a4194eb0919885454ff1f38f/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:52.779575 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a8e5cf-3411-4cec-be5d-87eb76341c8a None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:52.779575 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a8e5cf-3411-4cec-be5d-87eb76341c8a None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:52.779575 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a8e5cf-3411-4cec-be5d-87eb76341c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:52.780894 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e7632-77a0-49e5-929b-c53559fde505 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:52.781198 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e7632-77a0-49e5-929b-c53559fde505 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:52.781465 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e7632-77a0-49e5-929b-c53559fde505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:52.794940 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-71a8e5cf-3411-4cec-be5d-87eb76341c8a None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:52.796194 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-71a8e5cf-3411-4cec-be5d-87eb76341c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:52.809584 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-9b9e7632-77a0-49e5-929b-c53559fde505 None None] MFA Rules not processed for user `4cf79da3a4194eb0919885454ff1f38f`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:52.810756 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-9b9e7632-77a0-49e5-929b-c53559fde505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:52.812561 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:52.860335 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1346/2691] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:55:52.871166 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a8e5cf-3411-4cec-be5d-87eb76341c8a None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:52.872691 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-71a8e5cf-3411-4cec-be5d-87eb76341c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:52.882020 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1346/2692] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:55:52.886388 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9e7632-77a0-49e5-929b-c53559fde505 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:52.888120 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-9b9e7632-77a0-49e5-929b-c53559fde505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:52.901868 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9e7632-77a0-49e5-929b-c53559fde505 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:52.906487 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1392e3dd-b61e-49cf-b400-e95854339d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:52.907082 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392e3dd-b61e-49cf-b400-e95854339d51 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:52.907434 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392e3dd-b61e-49cf-b400-e95854339d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:52.907706 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392e3dd-b61e-49cf-b400-e95854339d51 admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:52.908522 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1392e3dd-b61e-49cf-b400-e95854339d51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:52.910561 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1392e3dd-b61e-49cf-b400-e95854339d51 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:52.937508 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a8e5cf-3411-4cec-be5d-87eb76341c8a None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:52.942027 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1347/2693] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:55:52 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:55:52.943025 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1392e3dd-b61e-49cf-b400-e95854339d51 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:52.989760 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c58ba-84c7-49b1-b606-7bc9858cc29b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:52.990625 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c58ba-84c7-49b1-b606-7bc9858cc29b service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:52.990981 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c58ba-84c7-49b1-b606-7bc9858cc29b service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:52.991253 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c58ba-84c7-49b1-b606-7bc9858cc29b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:52.996563 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:53.011206 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1392e3dd-b61e-49cf-b400-e95854339d51 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:53.017133 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429e61bc-a9ba-4cbd-bbf4-fd4f7b2f2b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:53.017731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e61bc-a9ba-4cbd-bbf4-fd4f7b2f2b2b admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:53.017960 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e61bc-a9ba-4cbd-bbf4-fd4f7b2f2b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:53.018238 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e61bc-a9ba-4cbd-bbf4-fd4f7b2f2b2b admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:53.018961 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429e61bc-a9ba-4cbd-bbf4-fd4f7b2f2b2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:53.021128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429e61bc-a9ba-4cbd-bbf4-fd4f7b2f2b2b admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:53.048212 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1348/2694] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:55:52 2022] POST /identity/v3/users => generated 298 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:55:53.052118 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429e61bc-a9ba-4cbd-bbf4-fd4f7b2f2b2b admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:53.059470 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9c58ba-84c7-49b1-b606-7bc9858cc29b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:53.060978 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9c58ba-84c7-49b1-b606-7bc9858cc29b service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:53.062047 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429e61bc-a9ba-4cbd-bbf4-fd4f7b2f2b2b admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:53.067564 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd70ea13-c462-42d6-8434-411dbb836eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:53.068249 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70ea13-c462-42d6-8434-411dbb836eba admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:53.068551 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70ea13-c462-42d6-8434-411dbb836eba admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:53.068878 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70ea13-c462-42d6-8434-411dbb836eba admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:53.069675 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd70ea13-c462-42d6-8434-411dbb836eba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:53.072056 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd70ea13-c462-42d6-8434-411dbb836eba admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:53.073099 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1347/2695] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:55:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:55:53.082907 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1349/2696] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:55:53.087702 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c58ba-84c7-49b1-b606-7bc9858cc29b service neutron] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:53.097741 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c58ba-84c7-49b1-b606-7bc9858cc29b service neutron] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:53.102831 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6df4e9-596b-4e63-807a-c8ad45efe745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:53.103516 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6df4e9-596b-4e63-807a-c8ad45efe745 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:53.103806 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6df4e9-596b-4e63-807a-c8ad45efe745 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:53.104246 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6df4e9-596b-4e63-807a-c8ad45efe745 admin admin] PATH_INFO: `/v3/projects/c7f577731fc447aaaeaba81b1fc25ec2/users/fa71ffc1217a46888c76e5524375faad/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:53.121955 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6df4e9-596b-4e63-807a-c8ad45efe745 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7f577731fc447aaaeaba81b1fc25ec2, user_id=fa71ffc1217a46888c76e5524375faad, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:53.125338 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6df4e9-596b-4e63-807a-c8ad45efe745 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:53.140145 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1348/2697] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:55:53 2022] PUT /identity/v3/projects/c7f577731fc447aaaeaba81b1fc25ec2/users/fa71ffc1217a46888c76e5524375faad/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:53.144020 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd70ea13-c462-42d6-8434-411dbb836eba admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:53.156557 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd70ea13-c462-42d6-8434-411dbb836eba admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:53.219333 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6a0eb-47cc-473b-a4b5-5afe4c897fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:53.220282 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6a0eb-47cc-473b-a4b5-5afe4c897fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:53.220372 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6a0eb-47cc-473b-a4b5-5afe4c897fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:53.220574 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6a0eb-47cc-473b-a4b5-5afe4c897fd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:53.221220 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b6a0eb-47cc-473b-a4b5-5afe4c897fd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:53.222930 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b6a0eb-47cc-473b-a4b5-5afe4c897fd8 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:53.233993 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1350/2698] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:55:53.238302 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6df4e9-596b-4e63-807a-c8ad45efe745 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:53.245360 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:53.249770 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6df4e9-596b-4e63-807a-c8ad45efe745 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:53.255980 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b8f70-d883-400f-a048-f7d6b5a7ff29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:53.256642 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b8f70-d883-400f-a048-f7d6b5a7ff29 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:53.256919 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b8f70-d883-400f-a048-f7d6b5a7ff29 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:53.257196 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b8f70-d883-400f-a048-f7d6b5a7ff29 admin admin] PATH_INFO: `/v3/projects/c7f577731fc447aaaeaba81b1fc25ec2/users/fa71ffc1217a46888c76e5524375faad/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:53.259761 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b8f70-d883-400f-a048-f7d6b5a7ff29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7f577731fc447aaaeaba81b1fc25ec2, user_id=fa71ffc1217a46888c76e5524375faad, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:53.262573 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b8f70-d883-400f-a048-f7d6b5a7ff29 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:53.273358 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1349/2699] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:55:53 2022] PUT /identity/v3/projects/c7f577731fc447aaaeaba81b1fc25ec2/users/fa71ffc1217a46888c76e5524375faad/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:55:53.284898 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26db11a-d138-42c0-9538-83e3846ebe5c None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:53.285177 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26db11a-d138-42c0-9538-83e3846ebe5c None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:53.285464 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26db11a-d138-42c0-9538-83e3846ebe5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:53.312640 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-d26db11a-d138-42c0-9538-83e3846ebe5c None None] MFA Rules not processed for user `fa71ffc1217a46888c76e5524375faad`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:53.313849 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-d26db11a-d138-42c0-9538-83e3846ebe5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:53.350023 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b8f70-d883-400f-a048-f7d6b5a7ff29 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:53.352212 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-5d7b8f70-d883-400f-a048-f7d6b5a7ff29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:53.387652 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1351/2700] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:53 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:55:53.400099 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26db11a-d138-42c0-9538-83e3846ebe5c None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:53.401560 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-d26db11a-d138-42c0-9538-83e3846ebe5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:53.438205 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b8f70-d883-400f-a048-f7d6b5a7ff29 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:53.451889 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd877364-d0eb-4e76-bb2b-31e73dd7069b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:53.451889 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd877364-d0eb-4e76-bb2b-31e73dd7069b admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:53.451889 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd877364-d0eb-4e76-bb2b-31e73dd7069b admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:53.451889 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd877364-d0eb-4e76-bb2b-31e73dd7069b admin admin] PATH_INFO: `/v3/projects/cab6ab8789e34c50b026a490df2f3a36` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:53.452415 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd877364-d0eb-4e76-bb2b-31e73dd7069b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cab6ab8789e34c50b026a490df2f3a36)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:53.452415 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd877364-d0eb-4e76-bb2b-31e73dd7069b admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:53.468151 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26db11a-d138-42c0-9538-83e3846ebe5c None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:53.472059 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd700db-65a9-420a-be41-1b3adb0ef549 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:53.472665 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd700db-65a9-420a-be41-1b3adb0ef549 service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:53.472970 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd700db-65a9-420a-be41-1b3adb0ef549 service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:53.473267 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd700db-65a9-420a-be41-1b3adb0ef549 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:53.479552 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:53.483521 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd700db-65a9-420a-be41-1b3adb0ef549 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:53.485579 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd700db-65a9-420a-be41-1b3adb0ef549 service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:53.495952 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1352/2701] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:55:53 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:55:53.500604 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-fd877364-d0eb-4e76-bb2b-31e73dd7069b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cab6ab8789e34c50b026a490df2f3a36', 'request_id': 'req-fd877364-d0eb-4e76-bb2b-31e73dd7069b'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:53.506387 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-fd877364-d0eb-4e76-bb2b-31e73dd7069b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cab6ab8789e34c50b026a490df2f3a36', 'request_id': 'req-fd877364-d0eb-4e76-bb2b-31e73dd7069b'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:53.514708 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-fd877364-d0eb-4e76-bb2b-31e73dd7069b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cab6ab8789e34c50b026a490df2f3a36', 'request_id': 'req-fd877364-d0eb-4e76-bb2b-31e73dd7069b'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:53.526849 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-fd877364-d0eb-4e76-bb2b-31e73dd7069b admin admin] The token cache is being invalidate because project cab6ab8789e34c50b026a490df2f3a36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:55:53.527195 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-fd877364-d0eb-4e76-bb2b-31e73dd7069b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:53.527866 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-fd877364-d0eb-4e76-bb2b-31e73dd7069b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:53.529450 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1350/2702] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:55:53 2022] DELETE /identity/v3/projects/cab6ab8789e34c50b026a490df2f3a36 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:55:53.534204 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb62e42-ac82-4228-b26a-c0d22c86b9af None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:53.534481 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb62e42-ac82-4228-b26a-c0d22c86b9af None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:53.534773 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb62e42-ac82-4228-b26a-c0d22c86b9af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:53.554518 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-bfb62e42-ac82-4228-b26a-c0d22c86b9af None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:53.556400 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-bfb62e42-ac82-4228-b26a-c0d22c86b9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:53.601917 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd877364-d0eb-4e76-bb2b-31e73dd7069b admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:53.604063 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-fd877364-d0eb-4e76-bb2b-31e73dd7069b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:53.626858 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1353/2703] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:55:53.638430 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0d0fd-5c87-4083-b88f-baf1a2dfe9db None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:53.638430 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0d0fd-5c87-4083-b88f-baf1a2dfe9db None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:53.638430 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0d0fd-5c87-4083-b88f-baf1a2dfe9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:53.658116 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-48d0d0fd-5c87-4083-b88f-baf1a2dfe9db None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:53.659342 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-48d0d0fd-5c87-4083-b88f-baf1a2dfe9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:53.670131 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1354/2704] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:55:53.676378 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0d0fd-5c87-4083-b88f-baf1a2dfe9db None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:53.679249 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd877364-d0eb-4e76-bb2b-31e73dd7069b admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:53.685459 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e95d453-8bb7-4cbc-ada4-a5f7382083ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:53.685459 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95d453-8bb7-4cbc-ada4-a5f7382083ca service cinder] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:53.685729 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95d453-8bb7-4cbc-ada4-a5f7382083ca service cinder] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:53.685844 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95d453-8bb7-4cbc-ada4-a5f7382083ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:53.686183 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0d0fd-5c87-4083-b88f-baf1a2dfe9db None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:53.687646 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-2e95d453-8bb7-4cbc-ada4-a5f7382083ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:53.690936 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4968573c-df3b-4750-9c1b-5c6b4070e160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:53.691528 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968573c-df3b-4750-9c1b-5c6b4070e160 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:53.691805 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968573c-df3b-4750-9c1b-5c6b4070e160 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:53.692108 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968573c-df3b-4750-9c1b-5c6b4070e160 admin admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:53.692757 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4968573c-df3b-4750-9c1b-5c6b4070e160 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:55:53.693058 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4968573c-df3b-4750-9c1b-5c6b4070e160 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:53.694731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4968573c-df3b-4750-9c1b-5c6b4070e160 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:53.708651 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1355/2705] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:55:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:55:53.712910 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4968573c-df3b-4750-9c1b-5c6b4070e160 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:53.728040 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4968573c-df3b-4750-9c1b-5c6b4070e160 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:53.728774 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:53.735023 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f70e9-7759-40e8-8b17-6319aaef585f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:53.735672 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f70e9-7759-40e8-8b17-6319aaef585f admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:53.735972 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f70e9-7759-40e8-8b17-6319aaef585f admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:53.736266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f70e9-7759-40e8-8b17-6319aaef585f admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:53.736909 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f70e9-7759-40e8-8b17-6319aaef585f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:53.738874 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f70e9-7759-40e8-8b17-6319aaef585f admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:53.764253 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e95d453-8bb7-4cbc-ada4-a5f7382083ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:53.765924 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e95d453-8bb7-4cbc-ada4-a5f7382083ca service cinder] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:53.770556 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1356/2706] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:55:53 2022] POST /identity/v3/projects => generated 391 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:55:53.774730 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f70e9-7759-40e8-8b17-6319aaef585f admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:53.777910 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1351/2707] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:55:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 23:55:53.854327 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f70e9-7759-40e8-8b17-6319aaef585f admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:53.861833 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe93dad8-dc36-4dd5-8bf7-bdf8f275e482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:53.862664 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe93dad8-dc36-4dd5-8bf7-bdf8f275e482 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:53.863056 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe93dad8-dc36-4dd5-8bf7-bdf8f275e482 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:53.863507 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe93dad8-dc36-4dd5-8bf7-bdf8f275e482 admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:53.864609 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe93dad8-dc36-4dd5-8bf7-bdf8f275e482 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:53.866987 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe93dad8-dc36-4dd5-8bf7-bdf8f275e482 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:53.890307 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e95d453-8bb7-4cbc-ada4-a5f7382083ca service cinder] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:53.891830 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-2e95d453-8bb7-4cbc-ada4-a5f7382083ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:53.895422 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1357/2708] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:55:53 2022] POST /identity/v3/users => generated 296 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:55:53.899462 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe93dad8-dc36-4dd5-8bf7-bdf8f275e482 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:53.909002 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e95d453-8bb7-4cbc-ada4-a5f7382083ca service cinder] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:53.910934 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe93dad8-dc36-4dd5-8bf7-bdf8f275e482 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:53.913901 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45782739-2944-4213-86a4-73f29f87dfcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:53.915123 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45782739-2944-4213-86a4-73f29f87dfcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:53.915466 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45782739-2944-4213-86a4-73f29f87dfcf admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:53.915799 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45782739-2944-4213-86a4-73f29f87dfcf admin admin] PATH_INFO: `/v3/users/4a93c76294e84aeba4fd8a0fa96b0774` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:53.917020 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103988c3-7130-4204-8fe5-834647fb34c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:53.917197 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45782739-2944-4213-86a4-73f29f87dfcf admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a93c76294e84aeba4fd8a0fa96b0774)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:53.917874 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103988c3-7130-4204-8fe5-834647fb34c6 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:53.918152 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103988c3-7130-4204-8fe5-834647fb34c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:53.918424 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103988c3-7130-4204-8fe5-834647fb34c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:53.919072 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103988c3-7130-4204-8fe5-834647fb34c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:53.919675 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45782739-2944-4213-86a4-73f29f87dfcf admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:53.920811 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103988c3-7130-4204-8fe5-834647fb34c6 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:53.936008 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1358/2709] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 23:55:53.937486 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103988c3-7130-4204-8fe5-834647fb34c6 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:53.948825 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103988c3-7130-4204-8fe5-834647fb34c6 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:53.953987 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e3dc0-e999-443a-b90b-a36e0bb3a3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:53.954602 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e3dc0-e999-443a-b90b-a36e0bb3a3e1 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:53.954846 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e3dc0-e999-443a-b90b-a36e0bb3a3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:53.955111 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e3dc0-e999-443a-b90b-a36e0bb3a3e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:53.955744 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e3dc0-e999-443a-b90b-a36e0bb3a3e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:53.957394 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e3dc0-e999-443a-b90b-a36e0bb3a3e1 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:53.971090 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1359/2710] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:55:53.975027 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e3dc0-e999-443a-b90b-a36e0bb3a3e1 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:53.979006 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-45782739-2944-4213-86a4-73f29f87dfcf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a93c76294e84aeba4fd8a0fa96b0774'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:53.979955 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-45782739-2944-4213-86a4-73f29f87dfcf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a93c76294e84aeba4fd8a0fa96b0774'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:53.980664 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-45782739-2944-4213-86a4-73f29f87dfcf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a93c76294e84aeba4fd8a0fa96b0774'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:53.987770 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e3dc0-e999-443a-b90b-a36e0bb3a3e1 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:53.989711 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-45782739-2944-4213-86a4-73f29f87dfcf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a93c76294e84aeba4fd8a0fa96b0774'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:53.993862 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b2c96-4257-46bc-8897-5d3c7801dd5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:53.994500 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b2c96-4257-46bc-8897-5d3c7801dd5f admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:53.994782 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b2c96-4257-46bc-8897-5d3c7801dd5f admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:53.995050 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b2c96-4257-46bc-8897-5d3c7801dd5f admin admin] PATH_INFO: `/v3/projects/8a55e99004364307b359d747336649fd/users/919c6627d8c14afdaa60920359f617f1/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:54.001595 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-45782739-2944-4213-86a4-73f29f87dfcf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a93c76294e84aeba4fd8a0fa96b0774'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:54.010465 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9b2c96-4257-46bc-8897-5d3c7801dd5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a55e99004364307b359d747336649fd, user_id=919c6627d8c14afdaa60920359f617f1, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:54.013909 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9b2c96-4257-46bc-8897-5d3c7801dd5f admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:54.017057 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:54.024235 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1352/2711] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:55:53 2022] DELETE /identity/v3/users/4a93c76294e84aeba4fd8a0fa96b0774 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:55:54.029792 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d16634-9f6f-45e1-916d-9b6040c2b5a3 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:54.030041 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d16634-9f6f-45e1-916d-9b6040c2b5a3 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:54.030407 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1360/2712] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:55:53 2022] PUT /identity/v3/projects/8a55e99004364307b359d747336649fd/users/919c6627d8c14afdaa60920359f617f1/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:55:54.030554 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d16634-9f6f-45e1-916d-9b6040c2b5a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:54.040856 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaaba98-8736-4a13-be9e-f9eca9a02efc None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:54.041103 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaaba98-8736-4a13-be9e-f9eca9a02efc None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:54.041379 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaaba98-8736-4a13-be9e-f9eca9a02efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:54.057693 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-96d16634-9f6f-45e1-916d-9b6040c2b5a3 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:54.059398 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-96d16634-9f6f-45e1-916d-9b6040c2b5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:54.080776 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-1aaaba98-8736-4a13-be9e-f9eca9a02efc None None] MFA Rules not processed for user `919c6627d8c14afdaa60920359f617f1`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:54.081981 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-1aaaba98-8736-4a13-be9e-f9eca9a02efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:54.150134 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1353/2713] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:55:54.159977 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1361/2714] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:54 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:55:54.161597 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d16634-9f6f-45e1-916d-9b6040c2b5a3 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:54.163063 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-96d16634-9f6f-45e1-916d-9b6040c2b5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:54.172902 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaaba98-8736-4a13-be9e-f9eca9a02efc None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:54.174702 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-1aaaba98-8736-4a13-be9e-f9eca9a02efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:54.222579 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:54.227616 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d16634-9f6f-45e1-916d-9b6040c2b5a3 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:54.232571 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3351eadb-c59c-4e52-9269-bff0c18a52e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:54.233394 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3351eadb-c59c-4e52-9269-bff0c18a52e7 service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:54.233873 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3351eadb-c59c-4e52-9269-bff0c18a52e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:54.234528 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3351eadb-c59c-4e52-9269-bff0c18a52e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:54.238117 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaaba98-8736-4a13-be9e-f9eca9a02efc None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:54.241750 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6142365c-609d-4369-997c-b1a95ad7f9b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:54.242374 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142365c-609d-4369-997c-b1a95ad7f9b4 service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:54.242652 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142365c-609d-4369-997c-b1a95ad7f9b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:54.242922 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142365c-609d-4369-997c-b1a95ad7f9b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:54.244344 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3351eadb-c59c-4e52-9269-bff0c18a52e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:54.245790 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3351eadb-c59c-4e52-9269-bff0c18a52e7 service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:54.251655 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6142365c-609d-4369-997c-b1a95ad7f9b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:54.253158 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6142365c-609d-4369-997c-b1a95ad7f9b4 service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:54.255337 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1354/2715] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:55:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:55:54.262262 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1362/2716] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:55:54 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:55:54.554905 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3351eadb-c59c-4e52-9269-bff0c18a52e7 service neutron] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:54.556509 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-3351eadb-c59c-4e52-9269-bff0c18a52e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:54.590820 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3351eadb-c59c-4e52-9269-bff0c18a52e7 service neutron] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:54.595790 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660902ac-b157-4511-a77c-832c0b2fe46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:54.596454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660902ac-b157-4511-a77c-832c0b2fe46b admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:54.596731 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660902ac-b157-4511-a77c-832c0b2fe46b admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:54.597000 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660902ac-b157-4511-a77c-832c0b2fe46b admin admin] PATH_INFO: `/v3/projects/4f72ce0661bf4104b3e4e53ed499813f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:54.598110 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660902ac-b157-4511-a77c-832c0b2fe46b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f72ce0661bf4104b3e4e53ed499813f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:54.600156 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660902ac-b157-4511-a77c-832c0b2fe46b admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:54.657894 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-660902ac-b157-4511-a77c-832c0b2fe46b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f72ce0661bf4104b3e4e53ed499813f', 'request_id': 'req-660902ac-b157-4511-a77c-832c0b2fe46b'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:54.662958 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-660902ac-b157-4511-a77c-832c0b2fe46b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f72ce0661bf4104b3e4e53ed499813f', 'request_id': 'req-660902ac-b157-4511-a77c-832c0b2fe46b'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:54.669935 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-660902ac-b157-4511-a77c-832c0b2fe46b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f72ce0661bf4104b3e4e53ed499813f', 'request_id': 'req-660902ac-b157-4511-a77c-832c0b2fe46b'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:54.680902 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-660902ac-b157-4511-a77c-832c0b2fe46b admin admin] The token cache is being invalidate because project 4f72ce0661bf4104b3e4e53ed499813f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:55:54.681184 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-660902ac-b157-4511-a77c-832c0b2fe46b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:54.681803 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-660902ac-b157-4511-a77c-832c0b2fe46b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:54.683281 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1355/2717] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:55:54 2022] DELETE /identity/v3/projects/4f72ce0661bf4104b3e4e53ed499813f => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:54.687555 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6142365c-609d-4369-997c-b1a95ad7f9b4 service cinder] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:54.698771 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6142365c-609d-4369-997c-b1a95ad7f9b4 service cinder] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:54.753383 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc43073b-a129-41f4-8f73-4cfe541e22b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:54.753995 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc43073b-a129-41f4-8f73-4cfe541e22b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:54.754270 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc43073b-a129-41f4-8f73-4cfe541e22b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:54.754540 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc43073b-a129-41f4-8f73-4cfe541e22b6 admin admin] PATH_INFO: `/v3/users/cbf2b089117841779810e2f9b57eeeaa` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:54.755649 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc43073b-a129-41f4-8f73-4cfe541e22b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbf2b089117841779810e2f9b57eeeaa)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:54.757597 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc43073b-a129-41f4-8f73-4cfe541e22b6 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:54.790110 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:54.803951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-fc43073b-a129-41f4-8f73-4cfe541e22b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbf2b089117841779810e2f9b57eeeaa'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:54.804629 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-fc43073b-a129-41f4-8f73-4cfe541e22b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbf2b089117841779810e2f9b57eeeaa'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:54.816311 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-fc43073b-a129-41f4-8f73-4cfe541e22b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbf2b089117841779810e2f9b57eeeaa'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:54.823792 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-fc43073b-a129-41f4-8f73-4cfe541e22b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbf2b089117841779810e2f9b57eeeaa'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:54.824693 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-fc43073b-a129-41f4-8f73-4cfe541e22b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbf2b089117841779810e2f9b57eeeaa'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:54.839067 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1363/2718] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:55:54 2022] DELETE /identity/v3/users/cbf2b089117841779810e2f9b57eeeaa => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:55.118832 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660902ac-b157-4511-a77c-832c0b2fe46b admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:55.120325 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-660902ac-b157-4511-a77c-832c0b2fe46b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:55.134660 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660902ac-b157-4511-a77c-832c0b2fe46b admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:55.203925 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5221cdc-8061-4bea-afd0-854baf112459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:55.204605 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5221cdc-8061-4bea-afd0-854baf112459 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:55.204889 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5221cdc-8061-4bea-afd0-854baf112459 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:55.205172 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5221cdc-8061-4bea-afd0-854baf112459 admin admin] PATH_INFO: `/v3/projects/524087aae8954053862e5a1c46b5bd1f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:55.206321 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5221cdc-8061-4bea-afd0-854baf112459 admin admin] RBAC: Authorizing `identity:delete_project(project_id=524087aae8954053862e5a1c46b5bd1f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:55.208320 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5221cdc-8061-4bea-afd0-854baf112459 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:55.255939 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-f5221cdc-8061-4bea-afd0-854baf112459 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '524087aae8954053862e5a1c46b5bd1f', 'request_id': 'req-f5221cdc-8061-4bea-afd0-854baf112459'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:55.262354 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-f5221cdc-8061-4bea-afd0-854baf112459 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '524087aae8954053862e5a1c46b5bd1f', 'request_id': 'req-f5221cdc-8061-4bea-afd0-854baf112459'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:55.270022 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-f5221cdc-8061-4bea-afd0-854baf112459 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '524087aae8954053862e5a1c46b5bd1f', 'request_id': 'req-f5221cdc-8061-4bea-afd0-854baf112459'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:55.282346 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-f5221cdc-8061-4bea-afd0-854baf112459 admin admin] The token cache is being invalidate because project 524087aae8954053862e5a1c46b5bd1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:55:55.282640 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-f5221cdc-8061-4bea-afd0-854baf112459 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:55.283293 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-f5221cdc-8061-4bea-afd0-854baf112459 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:55.285495 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1356/2719] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:55:55 2022] DELETE /identity/v3/projects/524087aae8954053862e5a1c46b5bd1f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:55.290827 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc81df-e41e-4a0b-a599-4ef16b3b8ecb None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:55.291329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc81df-e41e-4a0b-a599-4ef16b3b8ecb None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:55.291435 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc81df-e41e-4a0b-a599-4ef16b3b8ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:55.312971 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-79cc81df-e41e-4a0b-a599-4ef16b3b8ecb None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:55.314182 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-79cc81df-e41e-4a0b-a599-4ef16b3b8ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:55.377399 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1364/2720] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:55:55.386771 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca90b2d-a823-4db7-8851-1e3d93f6bdfe None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:55.387113 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca90b2d-a823-4db7-8851-1e3d93f6bdfe None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:55.387597 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca90b2d-a823-4db7-8851-1e3d93f6bdfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:55.396616 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:55.411582 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-4ca90b2d-a823-4db7-8851-1e3d93f6bdfe None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:55.412867 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-4ca90b2d-a823-4db7-8851-1e3d93f6bdfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:55.423016 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1357/2721] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:55:55.427520 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cc81df-e41e-4a0b-a599-4ef16b3b8ecb None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:55.437583 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cc81df-e41e-4a0b-a599-4ef16b3b8ecb None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:55.442162 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb1731-85ac-44c7-bd97-d28aba9f1149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:55.442757 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb1731-85ac-44c7-bd97-d28aba9f1149 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:55.443036 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb1731-85ac-44c7-bd97-d28aba9f1149 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:55.443305 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb1731-85ac-44c7-bd97-d28aba9f1149 admin admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:55.443978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb1731-85ac-44c7-bd97-d28aba9f1149 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:55:55.444369 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb1731-85ac-44c7-bd97-d28aba9f1149 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:55.445908 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb1731-85ac-44c7-bd97-d28aba9f1149 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:55.457513 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1365/2722] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:55:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:55:55.462011 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca90b2d-a823-4db7-8851-1e3d93f6bdfe None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:55.471663 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca90b2d-a823-4db7-8851-1e3d93f6bdfe None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:55.476310 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ac0c4-f84a-49a7-9808-02cf52eb5deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:55.477448 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ac0c4-f84a-49a7-9808-02cf52eb5deb admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:55.477706 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ac0c4-f84a-49a7-9808-02cf52eb5deb admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:55.477979 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ac0c4-f84a-49a7-9808-02cf52eb5deb admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:55.478597 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ac0c4-f84a-49a7-9808-02cf52eb5deb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:55.480619 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ac0c4-f84a-49a7-9808-02cf52eb5deb admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:55.507690 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1358/2723] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:55:55 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:55:55.511929 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb1731-85ac-44c7-bd97-d28aba9f1149 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:55.521842 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb1731-85ac-44c7-bd97-d28aba9f1149 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:55.577158 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff5d730-9191-4d63-b421-d307b987aa1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:55.577750 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5d730-9191-4d63-b421-d307b987aa1c admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:55.578024 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5d730-9191-4d63-b421-d307b987aa1c admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:55.578291 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5d730-9191-4d63-b421-d307b987aa1c admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:55.578923 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff5d730-9191-4d63-b421-d307b987aa1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:55.581157 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff5d730-9191-4d63-b421-d307b987aa1c admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:55.602982 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1366/2724] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:55:55 2022] POST /identity/v3/users => generated 284 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:55:55.607317 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ac0c4-f84a-49a7-9808-02cf52eb5deb admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:55.619088 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ac0c4-f84a-49a7-9808-02cf52eb5deb admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:55.624647 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3621fb08-bd92-44af-93a9-e6f54ae2efb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:55.625239 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621fb08-bd92-44af-93a9-e6f54ae2efb0 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:55.625514 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621fb08-bd92-44af-93a9-e6f54ae2efb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:55.626133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621fb08-bd92-44af-93a9-e6f54ae2efb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:55.626781 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3621fb08-bd92-44af-93a9-e6f54ae2efb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:55.628387 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3621fb08-bd92-44af-93a9-e6f54ae2efb0 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:55.639428 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1359/2725] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:55:55.643432 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff5d730-9191-4d63-b421-d307b987aa1c admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:55.654627 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff5d730-9191-4d63-b421-d307b987aa1c admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:55.659858 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce613a86-a9f1-428a-97d7-f2892ce30ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:55.660499 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce613a86-a9f1-428a-97d7-f2892ce30ebd admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:55.660775 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce613a86-a9f1-428a-97d7-f2892ce30ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:55.661045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce613a86-a9f1-428a-97d7-f2892ce30ebd admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:55.661691 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce613a86-a9f1-428a-97d7-f2892ce30ebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:55.663324 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce613a86-a9f1-428a-97d7-f2892ce30ebd admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:55.676483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1367/2726] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:55:55.681158 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3621fb08-bd92-44af-93a9-e6f54ae2efb0 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:55.691456 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3621fb08-bd92-44af-93a9-e6f54ae2efb0 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:55.696086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1835fe-4493-400e-b55f-237d36380d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:55.696728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1835fe-4493-400e-b55f-237d36380d7c admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:55.697005 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1835fe-4493-400e-b55f-237d36380d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:55.697274 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1835fe-4493-400e-b55f-237d36380d7c admin admin] PATH_INFO: `/v3/projects/0080cf5310764e4e9743fad74cf68196/users/6a1d237576934ea6891fd6c4428d5900/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:55.714194 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1835fe-4493-400e-b55f-237d36380d7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0080cf5310764e4e9743fad74cf68196, user_id=6a1d237576934ea6891fd6c4428d5900, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:55.717183 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1835fe-4493-400e-b55f-237d36380d7c admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:55.733619 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1360/2727] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:55:55 2022] PUT /identity/v3/projects/0080cf5310764e4e9743fad74cf68196/users/6a1d237576934ea6891fd6c4428d5900/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:55.744503 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603fae4-afe8-43a6-a8b8-216db7ad86be None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:55.744815 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603fae4-afe8-43a6-a8b8-216db7ad86be None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:55.745158 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603fae4-afe8-43a6-a8b8-216db7ad86be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:55.771521 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:55.775188 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-c603fae4-afe8-43a6-a8b8-216db7ad86be None None] MFA Rules not processed for user `6a1d237576934ea6891fd6c4428d5900`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:55.777401 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-c603fae4-afe8-43a6-a8b8-216db7ad86be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:55.853274 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1368/2728] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:55:55.857743 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1835fe-4493-400e-b55f-237d36380d7c admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:55.919908 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1835fe-4493-400e-b55f-237d36380d7c admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:55.924576 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89630f8e-0d34-4ec8-ada3-77fc3244e00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:55.925172 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89630f8e-0d34-4ec8-ada3-77fc3244e00a admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:55.925451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89630f8e-0d34-4ec8-ada3-77fc3244e00a admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:55.925727 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89630f8e-0d34-4ec8-ada3-77fc3244e00a admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:55.926387 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89630f8e-0d34-4ec8-ada3-77fc3244e00a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:55.928395 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89630f8e-0d34-4ec8-ada3-77fc3244e00a admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:55.954933 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1361/2729] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:55:55 2022] POST /identity/v3/projects => generated 369 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:55:55.958965 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c603fae4-afe8-43a6-a8b8-216db7ad86be None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:55.970014 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c603fae4-afe8-43a6-a8b8-216db7ad86be None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:56.024598 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71879d8f-bc33-4524-9d19-a057c3cfd297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:56.025200 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71879d8f-bc33-4524-9d19-a057c3cfd297 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:56.025478 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71879d8f-bc33-4524-9d19-a057c3cfd297 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:56.025745 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71879d8f-bc33-4524-9d19-a057c3cfd297 admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:56.026364 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71879d8f-bc33-4524-9d19-a057c3cfd297 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:56.028408 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71879d8f-bc33-4524-9d19-a057c3cfd297 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:56.048403 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1369/2730] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:55:55 2022] POST /identity/v3/users => generated 291 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:55:56.052349 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89630f8e-0d34-4ec8-ada3-77fc3244e00a admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:56.062614 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89630f8e-0d34-4ec8-ada3-77fc3244e00a admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:56.067219 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cfd009-a1fa-4ed4-8d1c-ff72b2548caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:56.067810 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfd009-a1fa-4ed4-8d1c-ff72b2548caa admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:56.068136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfd009-a1fa-4ed4-8d1c-ff72b2548caa admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:56.068414 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfd009-a1fa-4ed4-8d1c-ff72b2548caa admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:56.069057 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cfd009-a1fa-4ed4-8d1c-ff72b2548caa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:56.071303 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cfd009-a1fa-4ed4-8d1c-ff72b2548caa admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:56.081866 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1362/2731] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:55:56.086682 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71879d8f-bc33-4524-9d19-a057c3cfd297 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:56.096724 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71879d8f-bc33-4524-9d19-a057c3cfd297 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:56.101319 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99484c4-7b6e-42f2-bc8e-1fdc6cc82141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:56.102003 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99484c4-7b6e-42f2-bc8e-1fdc6cc82141 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:56.102295 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99484c4-7b6e-42f2-bc8e-1fdc6cc82141 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:56.102547 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99484c4-7b6e-42f2-bc8e-1fdc6cc82141 admin admin] PATH_INFO: `/v3/projects/821c0b607baf42d9a96c0270e48a2506/users/5d93a2d7ed754823b193098845a83ded/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:56.116691 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99484c4-7b6e-42f2-bc8e-1fdc6cc82141 admin admin] RBAC: Authorizing `identity:create_grant(project_id=821c0b607baf42d9a96c0270e48a2506, user_id=5d93a2d7ed754823b193098845a83ded, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:56.119346 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99484c4-7b6e-42f2-bc8e-1fdc6cc82141 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:56.131861 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1370/2732] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:55:56 2022] PUT /identity/v3/projects/821c0b607baf42d9a96c0270e48a2506/users/5d93a2d7ed754823b193098845a83ded/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:56.136204 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cfd009-a1fa-4ed4-8d1c-ff72b2548caa admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:56.146175 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cfd009-a1fa-4ed4-8d1c-ff72b2548caa admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:56.204975 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06035e-6b98-46dc-a95f-6c4fad99672a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:56.205528 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06035e-6b98-46dc-a95f-6c4fad99672a admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:56.205805 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06035e-6b98-46dc-a95f-6c4fad99672a admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:56.206074 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06035e-6b98-46dc-a95f-6c4fad99672a admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:56.206739 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe06035e-6b98-46dc-a95f-6c4fad99672a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:56.208409 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe06035e-6b98-46dc-a95f-6c4fad99672a admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:56.218511 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1363/2733] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:55:56.222640 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99484c4-7b6e-42f2-bc8e-1fdc6cc82141 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:56.232812 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99484c4-7b6e-42f2-bc8e-1fdc6cc82141 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:56.237586 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cdac9d-f10c-4a4a-a660-fc41a8251bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:56.238206 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cdac9d-f10c-4a4a-a660-fc41a8251bf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:56.238480 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cdac9d-f10c-4a4a-a660-fc41a8251bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:56.238751 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cdac9d-f10c-4a4a-a660-fc41a8251bf9 admin admin] PATH_INFO: `/v3/projects/821c0b607baf42d9a96c0270e48a2506/users/5d93a2d7ed754823b193098845a83ded/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:56.240653 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cdac9d-f10c-4a4a-a660-fc41a8251bf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=821c0b607baf42d9a96c0270e48a2506, user_id=5d93a2d7ed754823b193098845a83ded, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:56.243277 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cdac9d-f10c-4a4a-a660-fc41a8251bf9 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:56.253285 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1371/2734] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:55:56 2022] PUT /identity/v3/projects/821c0b607baf42d9a96c0270e48a2506/users/5d93a2d7ed754823b193098845a83ded/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:56.263893 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1fd65c-d433-4b81-8fe2-ca34a8f80e9a None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:56.264209 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1fd65c-d433-4b81-8fe2-ca34a8f80e9a None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:56.264484 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1fd65c-d433-4b81-8fe2-ca34a8f80e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:56.284611 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:56.293244 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-fb1fd65c-d433-4b81-8fe2-ca34a8f80e9a None None] MFA Rules not processed for user `5d93a2d7ed754823b193098845a83ded`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:56.294686 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-fb1fd65c-d433-4b81-8fe2-ca34a8f80e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:56.361232 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1364/2735] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:56 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:55:56.372642 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cdac9d-f10c-4a4a-a660-fc41a8251bf9 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:56.374506 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-c2cdac9d-f10c-4a4a-a660-fc41a8251bf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:56.431923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1fd65c-d433-4b81-8fe2-ca34a8f80e9a None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:56.433177 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cdac9d-f10c-4a4a-a660-fc41a8251bf9 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:56.433419 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-fb1fd65c-d433-4b81-8fe2-ca34a8f80e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:56.436882 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb7414a-9a64-45f3-9e64-492fd4b42591 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:56.437483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb7414a-9a64-45f3-9e64-492fd4b42591 service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:56.437759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb7414a-9a64-45f3-9e64-492fd4b42591 service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:56.438026 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb7414a-9a64-45f3-9e64-492fd4b42591 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:56.448441 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb7414a-9a64-45f3-9e64-492fd4b42591 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:56.449920 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb7414a-9a64-45f3-9e64-492fd4b42591 service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:56.454145 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:56.461998 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1372/2736] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:55:56 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:55:56.501123 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1fd65c-d433-4b81-8fe2-ca34a8f80e9a None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:56.505741 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ab460-0c59-4efa-bd66-cb8339f0b0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:56.506576 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ab460-0c59-4efa-bd66-cb8339f0b0e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:56.506859 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ab460-0c59-4efa-bd66-cb8339f0b0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:56.506995 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb7414a-9a64-45f3-9e64-492fd4b42591 service cinder] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:56.507135 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ab460-0c59-4efa-bd66-cb8339f0b0e8 admin admin] PATH_INFO: `/v3/users/4cf79da3a4194eb0919885454ff1f38f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:56.508838 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15ab460-0c59-4efa-bd66-cb8339f0b0e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cf79da3a4194eb0919885454ff1f38f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:56.509043 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-1bb7414a-9a64-45f3-9e64-492fd4b42591 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:56.510759 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15ab460-0c59-4efa-bd66-cb8339f0b0e8 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:56.520363 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb7414a-9a64-45f3-9e64-492fd4b42591 service cinder] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:56.524499 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9124bb2-9f04-47d7-9322-d86599ce47df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:56.525089 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9124bb2-9f04-47d7-9322-d86599ce47df service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:56.525366 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9124bb2-9f04-47d7-9322-d86599ce47df service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:56.525637 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9124bb2-9f04-47d7-9322-d86599ce47df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:56.559536 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-c15ab460-0c59-4efa-bd66-cb8339f0b0e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cf79da3a4194eb0919885454ff1f38f'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:56.560240 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-c15ab460-0c59-4efa-bd66-cb8339f0b0e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cf79da3a4194eb0919885454ff1f38f'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:56.560874 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-c15ab460-0c59-4efa-bd66-cb8339f0b0e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cf79da3a4194eb0919885454ff1f38f'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:56.568554 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-c15ab460-0c59-4efa-bd66-cb8339f0b0e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cf79da3a4194eb0919885454ff1f38f'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:56.576537 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-c15ab460-0c59-4efa-bd66-cb8339f0b0e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cf79da3a4194eb0919885454ff1f38f'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:56.592884 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9124bb2-9f04-47d7-9322-d86599ce47df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:56.596963 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1365/2737] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:55:56 2022] DELETE /identity/v3/users/4cf79da3a4194eb0919885454ff1f38f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:55:56.597107 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9124bb2-9f04-47d7-9322-d86599ce47df service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:56.598670 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd00433-82ab-41a5-ba24-e0356d7b6bfd None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:56.598957 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd00433-82ab-41a5-ba24-e0356d7b6bfd None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:56.599229 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd00433-82ab-41a5-ba24-e0356d7b6bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:56.603367 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1373/2738] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:55:56 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:55:56.618879 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-ccd00433-82ab-41a5-ba24-e0356d7b6bfd None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:56.620122 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-ccd00433-82ab-41a5-ba24-e0356d7b6bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:56.629555 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:56.685214 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1366/2739] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:55:56.695047 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9124bb2-9f04-47d7-9322-d86599ce47df service cinder] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:56.697150 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-f9124bb2-9f04-47d7-9322-d86599ce47df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:56.757633 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9124bb2-9f04-47d7-9322-d86599ce47df service cinder] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:56.808641 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e403cc-66ec-4447-a2b6-3edf9d3dafa4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:56.809230 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e403cc-66ec-4447-a2b6-3edf9d3dafa4 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:56.809564 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e403cc-66ec-4447-a2b6-3edf9d3dafa4 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:56.809902 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e403cc-66ec-4447-a2b6-3edf9d3dafa4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:56.820820 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e403cc-66ec-4447-a2b6-3edf9d3dafa4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:56.822610 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e403cc-66ec-4447-a2b6-3edf9d3dafa4 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:56.848473 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1374/2740] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:55:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:55:57.123954 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd00433-82ab-41a5-ba24-e0356d7b6bfd None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:57.125429 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-ccd00433-82ab-41a5-ba24-e0356d7b6bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:57.138578 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd00433-82ab-41a5-ba24-e0356d7b6bfd None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:57.143097 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f327d872-2dca-465e-8d5b-6e7b077808f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:57.143723 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327d872-2dca-465e-8d5b-6e7b077808f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:57.144022 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327d872-2dca-465e-8d5b-6e7b077808f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:57.144299 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327d872-2dca-465e-8d5b-6e7b077808f4 admin admin] PATH_INFO: `/v3/projects/37b7602604ce4244b6bfc2d5a6d26733` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:57.145422 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f327d872-2dca-465e-8d5b-6e7b077808f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37b7602604ce4244b6bfc2d5a6d26733)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:57.147418 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f327d872-2dca-465e-8d5b-6e7b077808f4 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:57.194780 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-f327d872-2dca-465e-8d5b-6e7b077808f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37b7602604ce4244b6bfc2d5a6d26733', 'request_id': 'req-f327d872-2dca-465e-8d5b-6e7b077808f4'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:57.200703 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-f327d872-2dca-465e-8d5b-6e7b077808f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37b7602604ce4244b6bfc2d5a6d26733', 'request_id': 'req-f327d872-2dca-465e-8d5b-6e7b077808f4'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:57.208056 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-f327d872-2dca-465e-8d5b-6e7b077808f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37b7602604ce4244b6bfc2d5a6d26733', 'request_id': 'req-f327d872-2dca-465e-8d5b-6e7b077808f4'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:57.219554 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-f327d872-2dca-465e-8d5b-6e7b077808f4 admin admin] The token cache is being invalidate because project 37b7602604ce4244b6bfc2d5a6d26733 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:55:57.219842 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-f327d872-2dca-465e-8d5b-6e7b077808f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:57.220548 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-f327d872-2dca-465e-8d5b-6e7b077808f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:57.222028 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1367/2741] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:55:57 2022] DELETE /identity/v3/projects/37b7602604ce4244b6bfc2d5a6d26733 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:57.226131 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e403cc-66ec-4447-a2b6-3edf9d3dafa4 service neutron] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:57.236356 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e403cc-66ec-4447-a2b6-3edf9d3dafa4 service neutron] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:57.284144 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:57.291902 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54049cc3-adc7-4507-ab12-413181388d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:57.292551 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54049cc3-adc7-4507-ab12-413181388d3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:57.293082 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54049cc3-adc7-4507-ab12-413181388d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:57.293353 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54049cc3-adc7-4507-ab12-413181388d3f admin admin] PATH_INFO: `/v3/users/fa71ffc1217a46888c76e5524375faad` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:57.294483 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54049cc3-adc7-4507-ab12-413181388d3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa71ffc1217a46888c76e5524375faad)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:57.296455 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54049cc3-adc7-4507-ab12-413181388d3f admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:57.341497 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-54049cc3-adc7-4507-ab12-413181388d3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa71ffc1217a46888c76e5524375faad'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:57.342156 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-54049cc3-adc7-4507-ab12-413181388d3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa71ffc1217a46888c76e5524375faad'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:57.354403 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-54049cc3-adc7-4507-ab12-413181388d3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa71ffc1217a46888c76e5524375faad'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:57.362449 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-54049cc3-adc7-4507-ab12-413181388d3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa71ffc1217a46888c76e5524375faad'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:57.363059 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-54049cc3-adc7-4507-ab12-413181388d3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa71ffc1217a46888c76e5524375faad'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:57.375006 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1375/2742] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:55:57 2022] DELETE /identity/v3/users/fa71ffc1217a46888c76e5524375faad => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:57.640341 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f327d872-2dca-465e-8d5b-6e7b077808f4 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:57.641876 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-f327d872-2dca-465e-8d5b-6e7b077808f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:57.660120 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f327d872-2dca-465e-8d5b-6e7b077808f4 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:57.716102 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee044b-0d56-4b28-9474-d07da2631c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:57.716757 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee044b-0d56-4b28-9474-d07da2631c49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:57.717036 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee044b-0d56-4b28-9474-d07da2631c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:57.717309 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee044b-0d56-4b28-9474-d07da2631c49 admin admin] PATH_INFO: `/v3/projects/c7f577731fc447aaaeaba81b1fc25ec2` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:57.718421 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eee044b-0d56-4b28-9474-d07da2631c49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7f577731fc447aaaeaba81b1fc25ec2)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:57.720377 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eee044b-0d56-4b28-9474-d07da2631c49 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:57.759372 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:57.766887 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-6eee044b-0d56-4b28-9474-d07da2631c49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7f577731fc447aaaeaba81b1fc25ec2', 'request_id': 'req-6eee044b-0d56-4b28-9474-d07da2631c49'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:57.772666 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-6eee044b-0d56-4b28-9474-d07da2631c49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7f577731fc447aaaeaba81b1fc25ec2', 'request_id': 'req-6eee044b-0d56-4b28-9474-d07da2631c49'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:57.780612 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-6eee044b-0d56-4b28-9474-d07da2631c49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7f577731fc447aaaeaba81b1fc25ec2', 'request_id': 'req-6eee044b-0d56-4b28-9474-d07da2631c49'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:57.797797 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-6eee044b-0d56-4b28-9474-d07da2631c49 admin admin] The token cache is being invalidate because project c7f577731fc447aaaeaba81b1fc25ec2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:55:57.798081 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-6eee044b-0d56-4b28-9474-d07da2631c49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:57.798716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-6eee044b-0d56-4b28-9474-d07da2631c49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:55:57.800220 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1368/2743] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:55:57 2022] DELETE /identity/v3/projects/c7f577731fc447aaaeaba81b1fc25ec2 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:57.805016 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeadbdba-0558-4c92-a2fa-9c26bcf02d79 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:57.805329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeadbdba-0558-4c92-a2fa-9c26bcf02d79 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:57.805601 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeadbdba-0558-4c92-a2fa-9c26bcf02d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:57.825443 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-eeadbdba-0558-4c92-a2fa-9c26bcf02d79 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:57.826559 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-eeadbdba-0558-4c92-a2fa-9c26bcf02d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:57.888532 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1376/2744] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:55:57.896291 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca987f7-bfa9-4aab-b402-a41830799553 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:57.896615 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca987f7-bfa9-4aab-b402-a41830799553 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:57.896845 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca987f7-bfa9-4aab-b402-a41830799553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:57.916870 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-aca987f7-bfa9-4aab-b402-a41830799553 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:57.918347 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-aca987f7-bfa9-4aab-b402-a41830799553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:57.926647 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1369/2745] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:55:57.931362 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeadbdba-0558-4c92-a2fa-9c26bcf02d79 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:57.941278 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeadbdba-0558-4c92-a2fa-9c26bcf02d79 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:57.946395 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50430c-2cc6-4df2-ab7f-0581ce7bce02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:57.946985 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50430c-2cc6-4df2-ab7f-0581ce7bce02 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:57.947288 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50430c-2cc6-4df2-ab7f-0581ce7bce02 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:57.947552 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50430c-2cc6-4df2-ab7f-0581ce7bce02 admin admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:57.948324 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd50430c-2cc6-4df2-ab7f-0581ce7bce02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:55:57.948620 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd50430c-2cc6-4df2-ab7f-0581ce7bce02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:57.950201 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd50430c-2cc6-4df2-ab7f-0581ce7bce02 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:57.961687 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1377/2746] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:55:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:55:57.967606 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca987f7-bfa9-4aab-b402-a41830799553 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:57.976815 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca987f7-bfa9-4aab-b402-a41830799553 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:57.981952 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e235146-79a2-4023-8bfc-5aa8b31e18f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:57.982556 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e235146-79a2-4023-8bfc-5aa8b31e18f0 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:57.982830 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e235146-79a2-4023-8bfc-5aa8b31e18f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:57.983199 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e235146-79a2-4023-8bfc-5aa8b31e18f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:57.983715 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e235146-79a2-4023-8bfc-5aa8b31e18f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:57.985632 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e235146-79a2-4023-8bfc-5aa8b31e18f0 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:58.018824 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1370/2747] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:55:57 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:55:58.020036 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50430c-2cc6-4df2-ab7f-0581ce7bce02 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:58.031471 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50430c-2cc6-4df2-ab7f-0581ce7bce02 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:58.097809 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc74d79b-abd1-4eb0-94ec-24cc550dc49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:58.098409 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74d79b-abd1-4eb0-94ec-24cc550dc49d admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:58.098687 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74d79b-abd1-4eb0-94ec-24cc550dc49d admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:58.098956 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74d79b-abd1-4eb0-94ec-24cc550dc49d admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:58.099626 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc74d79b-abd1-4eb0-94ec-24cc550dc49d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:58.101605 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc74d79b-abd1-4eb0-94ec-24cc550dc49d admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:58.110909 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:58.124349 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1378/2748] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:55:58 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:55:58.128260 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e235146-79a2-4023-8bfc-5aa8b31e18f0 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:58.139074 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e235146-79a2-4023-8bfc-5aa8b31e18f0 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:58.143709 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b019d9b-16da-498a-97f9-fb96218b4a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:58.144339 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b019d9b-16da-498a-97f9-fb96218b4a10 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:58.144620 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b019d9b-16da-498a-97f9-fb96218b4a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:58.144888 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b019d9b-16da-498a-97f9-fb96218b4a10 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:58.145527 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b019d9b-16da-498a-97f9-fb96218b4a10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:58.147137 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b019d9b-16da-498a-97f9-fb96218b4a10 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:58.160343 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1371/2749] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:55:58.161826 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc74d79b-abd1-4eb0-94ec-24cc550dc49d admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:58.172119 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc74d79b-abd1-4eb0-94ec-24cc550dc49d admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:58.176912 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9502be1e-493f-410a-80e8-105521cc4a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:58.177503 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502be1e-493f-410a-80e8-105521cc4a7d admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:58.177777 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502be1e-493f-410a-80e8-105521cc4a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:58.178044 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502be1e-493f-410a-80e8-105521cc4a7d admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:58.178682 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9502be1e-493f-410a-80e8-105521cc4a7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:58.180873 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9502be1e-493f-410a-80e8-105521cc4a7d admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:58.191651 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1379/2750] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:55:58.196593 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b019d9b-16da-498a-97f9-fb96218b4a10 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:58.206508 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b019d9b-16da-498a-97f9-fb96218b4a10 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:58.211946 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc9cc19-ac92-406f-b90c-2d1f607e4054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:58.212626 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9cc19-ac92-406f-b90c-2d1f607e4054 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:58.212910 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9cc19-ac92-406f-b90c-2d1f607e4054 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:58.213180 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9cc19-ac92-406f-b90c-2d1f607e4054 admin admin] PATH_INFO: `/v3/projects/66a8f78a096c4f72a806aae2ea820189/users/5bb3ba4bfb554e719c8f440698a17c4f/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:58.229464 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc9cc19-ac92-406f-b90c-2d1f607e4054 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66a8f78a096c4f72a806aae2ea820189, user_id=5bb3ba4bfb554e719c8f440698a17c4f, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:58.232237 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc9cc19-ac92-406f-b90c-2d1f607e4054 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:58.247311 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1372/2751] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:55:58 2022] PUT /identity/v3/projects/66a8f78a096c4f72a806aae2ea820189/users/5bb3ba4bfb554e719c8f440698a17c4f/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:55:58.259343 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c64dae-68a5-404b-865a-eeef7932ec71 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:58.259661 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c64dae-68a5-404b-865a-eeef7932ec71 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:58.260100 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c64dae-68a5-404b-865a-eeef7932ec71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:58.288225 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-d2c64dae-68a5-404b-865a-eeef7932ec71 None None] MFA Rules not processed for user `5bb3ba4bfb554e719c8f440698a17c4f`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:58.289967 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-d2c64dae-68a5-404b-865a-eeef7932ec71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:58.379106 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1380/2752] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:55:58.383415 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc9cc19-ac92-406f-b90c-2d1f607e4054 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:58.449938 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc9cc19-ac92-406f-b90c-2d1f607e4054 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:58.454797 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc57d2b-5ad0-4ac6-a49f-cbe6c123e619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:58.455404 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc57d2b-5ad0-4ac6-a49f-cbe6c123e619 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:58.455680 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc57d2b-5ad0-4ac6-a49f-cbe6c123e619 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:58.455949 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc57d2b-5ad0-4ac6-a49f-cbe6c123e619 admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:58.456628 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc57d2b-5ad0-4ac6-a49f-cbe6c123e619 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:58.458550 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc57d2b-5ad0-4ac6-a49f-cbe6c123e619 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:58.484717 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1373/2753] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:55:58 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:55:58.488973 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c64dae-68a5-404b-865a-eeef7932ec71 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:58.500465 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c64dae-68a5-404b-865a-eeef7932ec71 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:58.556106 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a038242-2549-483f-a9b5-d36b19d8a2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:58.556711 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a038242-2549-483f-a9b5-d36b19d8a2ac admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:58.557001 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a038242-2549-483f-a9b5-d36b19d8a2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:58.557259 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a038242-2549-483f-a9b5-d36b19d8a2ac admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:58.557896 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a038242-2549-483f-a9b5-d36b19d8a2ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:58.559806 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a038242-2549-483f-a9b5-d36b19d8a2ac admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:58.580753 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1381/2754] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:55:58 2022] POST /identity/v3/users => generated 298 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:55:58.584692 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc57d2b-5ad0-4ac6-a49f-cbe6c123e619 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:58.594960 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc57d2b-5ad0-4ac6-a49f-cbe6c123e619 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:58.599458 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b67328-9f3d-4d9e-91f7-faba9fd5e2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:58.600086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b67328-9f3d-4d9e-91f7-faba9fd5e2ec admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:58.600365 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b67328-9f3d-4d9e-91f7-faba9fd5e2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:58.600631 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b67328-9f3d-4d9e-91f7-faba9fd5e2ec admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:58.601396 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b67328-9f3d-4d9e-91f7-faba9fd5e2ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:58.603530 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b67328-9f3d-4d9e-91f7-faba9fd5e2ec admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:58.613818 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1374/2755] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:55:58.618569 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a038242-2549-483f-a9b5-d36b19d8a2ac admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:58.628352 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a038242-2549-483f-a9b5-d36b19d8a2ac admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:58.632892 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72093701-5a9e-44f6-a2d1-07f7146aa2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:58.633524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72093701-5a9e-44f6-a2d1-07f7146aa2fd admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:58.633802 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72093701-5a9e-44f6-a2d1-07f7146aa2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:58.634073 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72093701-5a9e-44f6-a2d1-07f7146aa2fd admin admin] PATH_INFO: `/v3/projects/fa951d5a9361411e8cc24bdaac5d70a5/users/1b2bcb5e8ad04fbb8af4364950fe6bcd/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:58.649540 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72093701-5a9e-44f6-a2d1-07f7146aa2fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa951d5a9361411e8cc24bdaac5d70a5, user_id=1b2bcb5e8ad04fbb8af4364950fe6bcd, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:58.652280 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72093701-5a9e-44f6-a2d1-07f7146aa2fd admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:58.664823 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1382/2756] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:55:58 2022] PUT /identity/v3/projects/fa951d5a9361411e8cc24bdaac5d70a5/users/1b2bcb5e8ad04fbb8af4364950fe6bcd/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:58.668931 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b67328-9f3d-4d9e-91f7-faba9fd5e2ec admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:58.679128 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b67328-9f3d-4d9e-91f7-faba9fd5e2ec admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:58.716420 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:58.736642 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853568ff-2301-4f0e-807d-e5be42a0d814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:58.737244 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853568ff-2301-4f0e-807d-e5be42a0d814 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:58.737541 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853568ff-2301-4f0e-807d-e5be42a0d814 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:58.737796 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853568ff-2301-4f0e-807d-e5be42a0d814 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:58.738437 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853568ff-2301-4f0e-807d-e5be42a0d814 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:58.740065 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853568ff-2301-4f0e-807d-e5be42a0d814 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:58.750156 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1375/2757] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 23:55:58.753751 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72093701-5a9e-44f6-a2d1-07f7146aa2fd admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:58.764097 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72093701-5a9e-44f6-a2d1-07f7146aa2fd admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:58.768704 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623d8117-0ce0-4305-8565-9190e007a9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:58.769339 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623d8117-0ce0-4305-8565-9190e007a9dc admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:58.769621 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623d8117-0ce0-4305-8565-9190e007a9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:58.769891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623d8117-0ce0-4305-8565-9190e007a9dc admin admin] PATH_INFO: `/v3/projects/fa951d5a9361411e8cc24bdaac5d70a5/users/1b2bcb5e8ad04fbb8af4364950fe6bcd/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:58.771929 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623d8117-0ce0-4305-8565-9190e007a9dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa951d5a9361411e8cc24bdaac5d70a5, user_id=1b2bcb5e8ad04fbb8af4364950fe6bcd, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:58.775128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623d8117-0ce0-4305-8565-9190e007a9dc admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:58.785215 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1383/2758] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:55:58 2022] PUT /identity/v3/projects/fa951d5a9361411e8cc24bdaac5d70a5/users/1b2bcb5e8ad04fbb8af4364950fe6bcd/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:55:58.850469 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b911b6a-5e05-4f57-ac22-bd2234a50b2c None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:58.850825 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b911b6a-5e05-4f57-ac22-bd2234a50b2c None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:58.851141 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b911b6a-5e05-4f57-ac22-bd2234a50b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:58.880169 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-2b911b6a-5e05-4f57-ac22-bd2234a50b2c None None] MFA Rules not processed for user `1b2bcb5e8ad04fbb8af4364950fe6bcd`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:55:58.881404 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-2b911b6a-5e05-4f57-ac22-bd2234a50b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:58.950437 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1376/2759] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:55:58 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:55:58.960891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623d8117-0ce0-4305-8565-9190e007a9dc admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:58.963101 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-623d8117-0ce0-4305-8565-9190e007a9dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:58.989743 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:55:59.021948 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623d8117-0ce0-4305-8565-9190e007a9dc admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:55:59.026410 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e580d6c-fe04-4850-a3bb-8d759d93cd9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:55:59.027012 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e580d6c-fe04-4850-a3bb-8d759d93cd9b service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:55:59.027295 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e580d6c-fe04-4850-a3bb-8d759d93cd9b service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:55:59.027595 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e580d6c-fe04-4850-a3bb-8d759d93cd9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:55:59.082759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e580d6c-fe04-4850-a3bb-8d759d93cd9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:55:59.084236 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e580d6c-fe04-4850-a3bb-8d759d93cd9b service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:55:59.093620 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1384/2760] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:55:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:55:59.931544 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b911b6a-5e05-4f57-ac22-bd2234a50b2c None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:55:59.932975 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-2b911b6a-5e05-4f57-ac22-bd2234a50b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:55:59.996354 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b911b6a-5e05-4f57-ac22-bd2234a50b2c None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:00.000809 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8de2a-628f-4017-9f49-8b87b9692c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:00.001430 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8de2a-628f-4017-9f49-8b87b9692c58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:00.001710 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8de2a-628f-4017-9f49-8b87b9692c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:00.001985 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8de2a-628f-4017-9f49-8b87b9692c58 admin admin] PATH_INFO: `/v3/users/6a1d237576934ea6891fd6c4428d5900` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:00.003091 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c8de2a-628f-4017-9f49-8b87b9692c58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a1d237576934ea6891fd6c4428d5900)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:00.005059 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c8de2a-628f-4017-9f49-8b87b9692c58 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:00.053426 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-c4c8de2a-628f-4017-9f49-8b87b9692c58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a1d237576934ea6891fd6c4428d5900'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:00.054081 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-c4c8de2a-628f-4017-9f49-8b87b9692c58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a1d237576934ea6891fd6c4428d5900'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:00.054763 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-c4c8de2a-628f-4017-9f49-8b87b9692c58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a1d237576934ea6891fd6c4428d5900'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:00.062534 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-c4c8de2a-628f-4017-9f49-8b87b9692c58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a1d237576934ea6891fd6c4428d5900'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:00.069690 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-c4c8de2a-628f-4017-9f49-8b87b9692c58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a1d237576934ea6891fd6c4428d5900'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:00.070802 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:56:00.089700 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1377/2761] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:55:59 2022] DELETE /identity/v3/users/6a1d237576934ea6891fd6c4428d5900 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:56:00.094141 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f5d2f-d9de-4240-aba2-512f113c123d None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:00.094615 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f5d2f-d9de-4240-aba2-512f113c123d None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:00.094749 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f5d2f-d9de-4240-aba2-512f113c123d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:00.113482 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-b16f5d2f-d9de-4240-aba2-512f113c123d None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:56:00.114666 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-b16f5d2f-d9de-4240-aba2-512f113c123d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:00.171930 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1385/2762] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:56:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:56:00.181514 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8de2a-628f-4017-9f49-8b87b9692c58 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:00.182938 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-c4c8de2a-628f-4017-9f49-8b87b9692c58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:00.363946 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8de2a-628f-4017-9f49-8b87b9692c58 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:00.367447 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36faeb8e-dfb5-42e3-b263-2de8d90c0a42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:00.368076 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36faeb8e-dfb5-42e3-b263-2de8d90c0a42 service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:00.368352 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36faeb8e-dfb5-42e3-b263-2de8d90c0a42 service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:00.368619 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36faeb8e-dfb5-42e3-b263-2de8d90c0a42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:00.377247 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36faeb8e-dfb5-42e3-b263-2de8d90c0a42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:00.378651 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36faeb8e-dfb5-42e3-b263-2de8d90c0a42 service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:00.388050 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1378/2763] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:56:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:56:00.656799 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16f5d2f-d9de-4240-aba2-512f113c123d None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:00.658232 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-b16f5d2f-d9de-4240-aba2-512f113c123d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:00.670887 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16f5d2f-d9de-4240-aba2-512f113c123d None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:00.675777 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63679ed6-5e20-44be-ad3d-f916726836ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:00.676434 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63679ed6-5e20-44be-ad3d-f916726836ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:00.676711 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63679ed6-5e20-44be-ad3d-f916726836ac admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:00.676985 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63679ed6-5e20-44be-ad3d-f916726836ac admin admin] PATH_INFO: `/v3/projects/0080cf5310764e4e9743fad74cf68196` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:00.678121 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63679ed6-5e20-44be-ad3d-f916726836ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=0080cf5310764e4e9743fad74cf68196)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:00.680120 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63679ed6-5e20-44be-ad3d-f916726836ac admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:00.727855 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-63679ed6-5e20-44be-ad3d-f916726836ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0080cf5310764e4e9743fad74cf68196', 'request_id': 'req-63679ed6-5e20-44be-ad3d-f916726836ac'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:00.733626 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-63679ed6-5e20-44be-ad3d-f916726836ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0080cf5310764e4e9743fad74cf68196', 'request_id': 'req-63679ed6-5e20-44be-ad3d-f916726836ac'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:00.740552 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-63679ed6-5e20-44be-ad3d-f916726836ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0080cf5310764e4e9743fad74cf68196', 'request_id': 'req-63679ed6-5e20-44be-ad3d-f916726836ac'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:00.748141 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:56:00.752364 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-63679ed6-5e20-44be-ad3d-f916726836ac admin admin] The token cache is being invalidate because project 0080cf5310764e4e9743fad74cf68196 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:56:00.752997 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-63679ed6-5e20-44be-ad3d-f916726836ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:00.753692 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-63679ed6-5e20-44be-ad3d-f916726836ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:00.755294 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1386/2764] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:56:00 2022] DELETE /identity/v3/projects/0080cf5310764e4e9743fad74cf68196 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:56:00.760136 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36faeb8e-dfb5-42e3-b263-2de8d90c0a42 service neutron] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:00.769642 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36faeb8e-dfb5-42e3-b263-2de8d90c0a42 service neutron] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:00.824074 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6ac9e-b347-4684-b634-3f5b35cdef7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:00.824709 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6ac9e-b347-4684-b634-3f5b35cdef7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:00.824993 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6ac9e-b347-4684-b634-3f5b35cdef7c admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:00.825272 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6ac9e-b347-4684-b634-3f5b35cdef7c admin admin] PATH_INFO: `/v3/users/5d93a2d7ed754823b193098845a83ded` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:00.826417 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d6ac9e-b347-4684-b634-3f5b35cdef7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d93a2d7ed754823b193098845a83ded)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:00.828402 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d6ac9e-b347-4684-b634-3f5b35cdef7c admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:00.871334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-76d6ac9e-b347-4684-b634-3f5b35cdef7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d93a2d7ed754823b193098845a83ded'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:00.871995 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-76d6ac9e-b347-4684-b634-3f5b35cdef7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d93a2d7ed754823b193098845a83ded'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:00.872590 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-76d6ac9e-b347-4684-b634-3f5b35cdef7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d93a2d7ed754823b193098845a83ded'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:00.879492 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-76d6ac9e-b347-4684-b634-3f5b35cdef7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d93a2d7ed754823b193098845a83ded'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:00.886765 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-76d6ac9e-b347-4684-b634-3f5b35cdef7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d93a2d7ed754823b193098845a83ded'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:00.904447 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1379/2765] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:56:00 2022] DELETE /identity/v3/users/5d93a2d7ed754823b193098845a83ded => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:56:01.196388 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63679ed6-5e20-44be-ad3d-f916726836ac admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:01.197824 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-63679ed6-5e20-44be-ad3d-f916726836ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:01.210648 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63679ed6-5e20-44be-ad3d-f916726836ac admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:01.268500 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de96fac5-7a7f-49ca-8d68-0ea9b70887d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:01.269049 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96fac5-7a7f-49ca-8d68-0ea9b70887d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:01.269330 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96fac5-7a7f-49ca-8d68-0ea9b70887d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:01.269598 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96fac5-7a7f-49ca-8d68-0ea9b70887d1 admin admin] PATH_INFO: `/v3/projects/821c0b607baf42d9a96c0270e48a2506` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:01.270698 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de96fac5-7a7f-49ca-8d68-0ea9b70887d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=821c0b607baf42d9a96c0270e48a2506)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:01.272628 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de96fac5-7a7f-49ca-8d68-0ea9b70887d1 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:01.318317 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-de96fac5-7a7f-49ca-8d68-0ea9b70887d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '821c0b607baf42d9a96c0270e48a2506', 'request_id': 'req-de96fac5-7a7f-49ca-8d68-0ea9b70887d1'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:01.323803 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-de96fac5-7a7f-49ca-8d68-0ea9b70887d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '821c0b607baf42d9a96c0270e48a2506', 'request_id': 'req-de96fac5-7a7f-49ca-8d68-0ea9b70887d1'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:01.330733 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-de96fac5-7a7f-49ca-8d68-0ea9b70887d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '821c0b607baf42d9a96c0270e48a2506', 'request_id': 'req-de96fac5-7a7f-49ca-8d68-0ea9b70887d1'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:01.341740 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-de96fac5-7a7f-49ca-8d68-0ea9b70887d1 admin admin] The token cache is being invalidate because project 821c0b607baf42d9a96c0270e48a2506 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:56:01.342030 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-de96fac5-7a7f-49ca-8d68-0ea9b70887d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:01.342660 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-de96fac5-7a7f-49ca-8d68-0ea9b70887d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:01.344197 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1387/2766] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:56:01 2022] DELETE /identity/v3/projects/821c0b607baf42d9a96c0270e48a2506 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:56:01.349042 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de692c-3274-4de9-937e-eff500df6486 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:01.349533 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de692c-3274-4de9-937e-eff500df6486 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:01.349657 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de692c-3274-4de9-937e-eff500df6486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:01.357422 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:56:01.371610 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-32de692c-3274-4de9-937e-eff500df6486 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:56:01.372946 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-32de692c-3274-4de9-937e-eff500df6486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:01.428710 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1380/2767] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:56:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:56:01.436931 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf71d01-d60a-4ce2-8747-5cff2d6b7f66 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:01.437210 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf71d01-d60a-4ce2-8747-5cff2d6b7f66 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:01.437474 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf71d01-d60a-4ce2-8747-5cff2d6b7f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:01.456572 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-bdf71d01-d60a-4ce2-8747-5cff2d6b7f66 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:56:01.457755 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-bdf71d01-d60a-4ce2-8747-5cff2d6b7f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:01.465546 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1388/2768] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:56:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:56:01.469830 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de692c-3274-4de9-937e-eff500df6486 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:01.480196 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de692c-3274-4de9-937e-eff500df6486 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:01.484723 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d685076-ee77-4583-96ed-b0f38e9b2ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:01.485341 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d685076-ee77-4583-96ed-b0f38e9b2ca8 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:01.485618 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d685076-ee77-4583-96ed-b0f38e9b2ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:01.485883 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d685076-ee77-4583-96ed-b0f38e9b2ca8 admin admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:01.486522 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d685076-ee77-4583-96ed-b0f38e9b2ca8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:56:01.486819 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d685076-ee77-4583-96ed-b0f38e9b2ca8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:01.488441 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d685076-ee77-4583-96ed-b0f38e9b2ca8 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:01.500621 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1381/2769] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:56:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:56:01.505731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf71d01-d60a-4ce2-8747-5cff2d6b7f66 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:01.515215 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf71d01-d60a-4ce2-8747-5cff2d6b7f66 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:01.519823 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5ee9e-4a78-40ee-944c-0c28b803b90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:01.520386 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5ee9e-4a78-40ee-944c-0c28b803b90e admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:01.520661 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5ee9e-4a78-40ee-944c-0c28b803b90e admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:01.520926 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5ee9e-4a78-40ee-944c-0c28b803b90e admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:01.521539 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d5ee9e-4a78-40ee-944c-0c28b803b90e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:01.523697 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d5ee9e-4a78-40ee-944c-0c28b803b90e admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:01.555995 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1389/2770] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:56:01 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:56:01.559445 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d685076-ee77-4583-96ed-b0f38e9b2ca8 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:01.570152 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d685076-ee77-4583-96ed-b0f38e9b2ca8 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:01.627448 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2bc5cc-f161-4448-9fcb-714a25be3319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:01.628090 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2bc5cc-f161-4448-9fcb-714a25be3319 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:01.628370 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2bc5cc-f161-4448-9fcb-714a25be3319 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:01.628640 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2bc5cc-f161-4448-9fcb-714a25be3319 admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:01.629255 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2bc5cc-f161-4448-9fcb-714a25be3319 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:01.631200 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2bc5cc-f161-4448-9fcb-714a25be3319 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:01.653133 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1382/2771] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:56:01 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:56:01.657287 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5ee9e-4a78-40ee-944c-0c28b803b90e admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:01.668859 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5ee9e-4a78-40ee-944c-0c28b803b90e admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:01.674083 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197fa440-ba08-4098-96e9-cb7dbb9b5d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:01.674682 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fa440-ba08-4098-96e9-cb7dbb9b5d1d admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:01.674959 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fa440-ba08-4098-96e9-cb7dbb9b5d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:01.675229 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fa440-ba08-4098-96e9-cb7dbb9b5d1d admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:01.675875 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197fa440-ba08-4098-96e9-cb7dbb9b5d1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:01.677504 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197fa440-ba08-4098-96e9-cb7dbb9b5d1d admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:01.687572 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1390/2772] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 23:56:01.691591 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2bc5cc-f161-4448-9fcb-714a25be3319 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:01.701638 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2bc5cc-f161-4448-9fcb-714a25be3319 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:01.706415 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f418e24-6d1e-4b38-85fc-aa97772f9a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:01.707003 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f418e24-6d1e-4b38-85fc-aa97772f9a80 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:01.707281 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f418e24-6d1e-4b38-85fc-aa97772f9a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:01.707552 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f418e24-6d1e-4b38-85fc-aa97772f9a80 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:01.708233 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f418e24-6d1e-4b38-85fc-aa97772f9a80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:01.709835 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f418e24-6d1e-4b38-85fc-aa97772f9a80 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:01.723165 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1383/2773] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:56:01.724554 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197fa440-ba08-4098-96e9-cb7dbb9b5d1d admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:01.735323 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197fa440-ba08-4098-96e9-cb7dbb9b5d1d admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:01.740182 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb84b1-6238-4827-9396-1cdc33fb8333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:01.740827 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb84b1-6238-4827-9396-1cdc33fb8333 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:01.741116 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb84b1-6238-4827-9396-1cdc33fb8333 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:01.741371 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb84b1-6238-4827-9396-1cdc33fb8333 admin admin] PATH_INFO: `/v3/projects/adca35ce4ab54a7284a8282851c9623e/users/8b1b4cc85bda4ffe948131a8b7113ada/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:01.758871 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb84b1-6238-4827-9396-1cdc33fb8333 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adca35ce4ab54a7284a8282851c9623e, user_id=8b1b4cc85bda4ffe948131a8b7113ada, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:01.761696 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb84b1-6238-4827-9396-1cdc33fb8333 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:01.775392 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1391/2774] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:56:01 2022] PUT /identity/v3/projects/adca35ce4ab54a7284a8282851c9623e/users/8b1b4cc85bda4ffe948131a8b7113ada/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:56:01.785667 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12318e90-3f19-4a2b-aca7-1522f8c10f66 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:01.785977 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12318e90-3f19-4a2b-aca7-1522f8c10f66 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:01.786250 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12318e90-3f19-4a2b-aca7-1522f8c10f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:01.825460 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-12318e90-3f19-4a2b-aca7-1522f8c10f66 None None] MFA Rules not processed for user `8b1b4cc85bda4ffe948131a8b7113ada`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:56:01.828374 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-12318e90-3f19-4a2b-aca7-1522f8c10f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:01.897179 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:56:01.910499 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1384/2775] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:56:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:56:01.911606 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb84b1-6238-4827-9396-1cdc33fb8333 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:01.942702 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:56:01.977250 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb84b1-6238-4827-9396-1cdc33fb8333 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:01.981863 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29923cf-c35d-4749-b53c-d9a561a80e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:01.982455 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29923cf-c35d-4749-b53c-d9a561a80e84 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:01.982773 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29923cf-c35d-4749-b53c-d9a561a80e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:01.983026 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29923cf-c35d-4749-b53c-d9a561a80e84 admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:01.983684 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29923cf-c35d-4749-b53c-d9a561a80e84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:01.985616 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29923cf-c35d-4749-b53c-d9a561a80e84 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:02.011254 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1392/2776] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:56:01 2022] POST /identity/v3/projects => generated 379 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:56:02.015455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12318e90-3f19-4a2b-aca7-1522f8c10f66 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:02.026588 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12318e90-3f19-4a2b-aca7-1522f8c10f66 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:02.079846 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fee42b6-384b-4f58-94ef-de2fefb227fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:02.080470 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee42b6-384b-4f58-94ef-de2fefb227fd admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:02.080746 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee42b6-384b-4f58-94ef-de2fefb227fd admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:02.081014 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee42b6-384b-4f58-94ef-de2fefb227fd admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:02.081620 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fee42b6-384b-4f58-94ef-de2fefb227fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:02.083509 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fee42b6-384b-4f58-94ef-de2fefb227fd admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:02.103747 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1385/2777] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:56:02 2022] POST /identity/v3/users => generated 290 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:56:02.107785 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29923cf-c35d-4749-b53c-d9a561a80e84 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:02.117987 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29923cf-c35d-4749-b53c-d9a561a80e84 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:02.122546 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911194f1-2afc-4e78-8164-0506bbb0113d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:02.123156 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911194f1-2afc-4e78-8164-0506bbb0113d admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:02.123431 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911194f1-2afc-4e78-8164-0506bbb0113d admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:02.123700 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911194f1-2afc-4e78-8164-0506bbb0113d admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:02.124382 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911194f1-2afc-4e78-8164-0506bbb0113d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:02.125964 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911194f1-2afc-4e78-8164-0506bbb0113d admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:02.139843 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1393/2778] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:56:02.141019 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fee42b6-384b-4f58-94ef-de2fefb227fd admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:02.151495 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fee42b6-384b-4f58-94ef-de2fefb227fd admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:02.160618 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80edea4d-eca9-479a-a174-a9d71bdc9c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:02.161221 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80edea4d-eca9-479a-a174-a9d71bdc9c35 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:02.161496 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80edea4d-eca9-479a-a174-a9d71bdc9c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:02.161766 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80edea4d-eca9-479a-a174-a9d71bdc9c35 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:02.162409 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80edea4d-eca9-479a-a174-a9d71bdc9c35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:02.164007 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80edea4d-eca9-479a-a174-a9d71bdc9c35 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:02.174105 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1386/2779] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:56:02.178130 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911194f1-2afc-4e78-8164-0506bbb0113d admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:02.187889 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911194f1-2afc-4e78-8164-0506bbb0113d admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:02.192453 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0239705d-c94c-4c51-b07a-ab347276c88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:02.193086 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0239705d-c94c-4c51-b07a-ab347276c88c admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:02.193366 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0239705d-c94c-4c51-b07a-ab347276c88c admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:02.193637 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0239705d-c94c-4c51-b07a-ab347276c88c admin admin] PATH_INFO: `/v3/projects/cd576de14bab4393848f2299c21f61f3/users/796d115d08a84bb98ec30d6a2cc7e459/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:02.208112 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0239705d-c94c-4c51-b07a-ab347276c88c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd576de14bab4393848f2299c21f61f3, user_id=796d115d08a84bb98ec30d6a2cc7e459, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:02.210747 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0239705d-c94c-4c51-b07a-ab347276c88c admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:02.223956 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1394/2780] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:56:02 2022] PUT /identity/v3/projects/cd576de14bab4393848f2299c21f61f3/users/796d115d08a84bb98ec30d6a2cc7e459/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:56:02.235247 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64acffb-979e-460f-b833-f044e112bbc5 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:02.235561 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64acffb-979e-460f-b833-f044e112bbc5 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:02.235859 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64acffb-979e-460f-b833-f044e112bbc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:02.262744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-a64acffb-979e-460f-b833-f044e112bbc5 None None] MFA Rules not processed for user `796d115d08a84bb98ec30d6a2cc7e459`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:56:02.263929 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-a64acffb-979e-460f-b833-f044e112bbc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:02.325698 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1387/2781] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:56:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:56:02.330036 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0239705d-c94c-4c51-b07a-ab347276c88c admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:02.392656 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0239705d-c94c-4c51-b07a-ab347276c88c admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:02.397157 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2cbad8-c7de-42c4-87d5-ebae16f136c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:02.397731 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2cbad8-c7de-42c4-87d5-ebae16f136c9 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:02.398004 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2cbad8-c7de-42c4-87d5-ebae16f136c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:02.398272 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2cbad8-c7de-42c4-87d5-ebae16f136c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:02.399448 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2cbad8-c7de-42c4-87d5-ebae16f136c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:02.401390 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2cbad8-c7de-42c4-87d5-ebae16f136c9 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:02.427219 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1395/2782] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:56:02 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:56:02.431807 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64acffb-979e-460f-b833-f044e112bbc5 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:02.444294 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64acffb-979e-460f-b833-f044e112bbc5 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:02.502046 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6256a71-533e-445a-98e6-ab61305c6f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:02.502646 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6256a71-533e-445a-98e6-ab61305c6f0d admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:02.502922 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6256a71-533e-445a-98e6-ab61305c6f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:02.503191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6256a71-533e-445a-98e6-ab61305c6f0d admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:02.503806 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6256a71-533e-445a-98e6-ab61305c6f0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:02.505735 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6256a71-533e-445a-98e6-ab61305c6f0d admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:02.516722 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:56:02.527674 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1388/2783] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:56:02 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:56:02.531683 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2cbad8-c7de-42c4-87d5-ebae16f136c9 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:02.542650 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2cbad8-c7de-42c4-87d5-ebae16f136c9 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:02.547477 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff5ec2-d1a6-48f3-9ad1-d32a38ea71ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:02.548117 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff5ec2-d1a6-48f3-9ad1-d32a38ea71ff admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:02.548393 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff5ec2-d1a6-48f3-9ad1-d32a38ea71ff admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:02.548657 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff5ec2-d1a6-48f3-9ad1-d32a38ea71ff admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:02.549302 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ff5ec2-d1a6-48f3-9ad1-d32a38ea71ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:02.550876 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ff5ec2-d1a6-48f3-9ad1-d32a38ea71ff admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:02.561696 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1396/2784] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:56:02.565746 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6256a71-533e-445a-98e6-ab61305c6f0d admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:02.576132 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6256a71-533e-445a-98e6-ab61305c6f0d admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:02.581031 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f5e6b-51bd-4475-b222-d5cb1316b5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:02.581674 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f5e6b-51bd-4475-b222-d5cb1316b5d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:02.581954 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f5e6b-51bd-4475-b222-d5cb1316b5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:02.582223 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f5e6b-51bd-4475-b222-d5cb1316b5d0 admin admin] PATH_INFO: `/v3/projects/919f80553b1f4995a574c3de3541502b/users/544c550d7aec41af9c6bfc41188b3a4d/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:02.596515 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45f5e6b-51bd-4475-b222-d5cb1316b5d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=919f80553b1f4995a574c3de3541502b, user_id=544c550d7aec41af9c6bfc41188b3a4d, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:02.599151 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45f5e6b-51bd-4475-b222-d5cb1316b5d0 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:02.612338 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1389/2785] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:56:02 2022] PUT /identity/v3/projects/919f80553b1f4995a574c3de3541502b/users/544c550d7aec41af9c6bfc41188b3a4d/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:56:02.616981 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff5ec2-d1a6-48f3-9ad1-d32a38ea71ff admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:02.627395 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff5ec2-d1a6-48f3-9ad1-d32a38ea71ff admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:02.672779 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:56:02.683556 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a66d1a-5bd2-407c-b316-40ade8d908e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:02.684191 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a66d1a-5bd2-407c-b316-40ade8d908e4 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:02.684455 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a66d1a-5bd2-407c-b316-40ade8d908e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:02.684721 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a66d1a-5bd2-407c-b316-40ade8d908e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:02.685354 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a66d1a-5bd2-407c-b316-40ade8d908e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:02.686951 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a66d1a-5bd2-407c-b316-40ade8d908e4 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:02.696955 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1397/2786] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 23:56:02.701018 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f5e6b-51bd-4475-b222-d5cb1316b5d0 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:02.711037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f5e6b-51bd-4475-b222-d5cb1316b5d0 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:02.715485 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce7260-25e5-48e8-bfd0-4d5c2e41a351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:02.716146 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce7260-25e5-48e8-bfd0-4d5c2e41a351 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:02.716421 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce7260-25e5-48e8-bfd0-4d5c2e41a351 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:02.716689 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce7260-25e5-48e8-bfd0-4d5c2e41a351 admin admin] PATH_INFO: `/v3/projects/919f80553b1f4995a574c3de3541502b/users/544c550d7aec41af9c6bfc41188b3a4d/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:02.718576 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce7260-25e5-48e8-bfd0-4d5c2e41a351 admin admin] RBAC: Authorizing `identity:create_grant(project_id=919f80553b1f4995a574c3de3541502b, user_id=544c550d7aec41af9c6bfc41188b3a4d, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:02.721243 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce7260-25e5-48e8-bfd0-4d5c2e41a351 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:02.730992 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1390/2787] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:56:02 2022] PUT /identity/v3/projects/919f80553b1f4995a574c3de3541502b/users/544c550d7aec41af9c6bfc41188b3a4d/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 30 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:56:02.740993 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ae95e-8c9c-4b41-8aa4-62a65186deb3 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:02.741280 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ae95e-8c9c-4b41-8aa4-62a65186deb3 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:02.741554 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ae95e-8c9c-4b41-8aa4-62a65186deb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:02.770098 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-b86ae95e-8c9c-4b41-8aa4-62a65186deb3 None None] MFA Rules not processed for user `544c550d7aec41af9c6bfc41188b3a4d`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:56:02.771301 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-b86ae95e-8c9c-4b41-8aa4-62a65186deb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:02.836741 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1398/2788] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:56:02 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:56:02.847884 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce7260-25e5-48e8-bfd0-4d5c2e41a351 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:02.849582 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-7cce7260-25e5-48e8-bfd0-4d5c2e41a351 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:02.907224 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce7260-25e5-48e8-bfd0-4d5c2e41a351 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:02.910812 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2872909-ce83-435e-b74c-e2954ae992ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:02.911413 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2872909-ce83-435e-b74c-e2954ae992ba service cinder] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:02.911690 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2872909-ce83-435e-b74c-e2954ae992ba service cinder] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:02.911972 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2872909-ce83-435e-b74c-e2954ae992ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:02.921691 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2872909-ce83-435e-b74c-e2954ae992ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:02.923112 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2872909-ce83-435e-b74c-e2954ae992ba service cinder] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:02.933086 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1391/2789] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:56:02 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:56:02.972405 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ae95e-8c9c-4b41-8aa4-62a65186deb3 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:02.973829 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-b86ae95e-8c9c-4b41-8aa4-62a65186deb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:02.985965 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ae95e-8c9c-4b41-8aa4-62a65186deb3 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:02.989767 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e667c7ee-e3eb-4d37-b4f3-77dca800370a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:02.990368 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667c7ee-e3eb-4d37-b4f3-77dca800370a service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:02.990644 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667c7ee-e3eb-4d37-b4f3-77dca800370a service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:02.990913 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667c7ee-e3eb-4d37-b4f3-77dca800370a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:03.046000 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e667c7ee-e3eb-4d37-b4f3-77dca800370a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:03.047529 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e667c7ee-e3eb-4d37-b4f3-77dca800370a service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:03.057117 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1399/2790] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:56:02 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 23:56:12.928751 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=43,keystone:INSERT=2 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:56:13.052413 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=68 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:56:13.266392 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2872909-ce83-435e-b74c-e2954ae992ba service cinder] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:13.332706 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2872909-ce83-435e-b74c-e2954ae992ba service cinder] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:13.337945 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e13dd3-95a9-4cf4-9ea1-bb6f9b25e8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:13.338574 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e13dd3-95a9-4cf4-9ea1-bb6f9b25e8c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:13.338848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e13dd3-95a9-4cf4-9ea1-bb6f9b25e8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:13.339118 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e13dd3-95a9-4cf4-9ea1-bb6f9b25e8c7 admin admin] PATH_INFO: `/v3/users/8b1b4cc85bda4ffe948131a8b7113ada` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:13.340242 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e13dd3-95a9-4cf4-9ea1-bb6f9b25e8c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b1b4cc85bda4ffe948131a8b7113ada)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:13.342191 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e13dd3-95a9-4cf4-9ea1-bb6f9b25e8c7 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:13.387871 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-c1e13dd3-95a9-4cf4-9ea1-bb6f9b25e8c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b1b4cc85bda4ffe948131a8b7113ada'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:13.388548 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-c1e13dd3-95a9-4cf4-9ea1-bb6f9b25e8c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b1b4cc85bda4ffe948131a8b7113ada'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:13.389153 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-c1e13dd3-95a9-4cf4-9ea1-bb6f9b25e8c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b1b4cc85bda4ffe948131a8b7113ada'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:13.396279 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-c1e13dd3-95a9-4cf4-9ea1-bb6f9b25e8c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b1b4cc85bda4ffe948131a8b7113ada'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:13.403924 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-c1e13dd3-95a9-4cf4-9ea1-bb6f9b25e8c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b1b4cc85bda4ffe948131a8b7113ada'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:13.420750 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1392/2791] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:56:13 2022] DELETE /identity/v3/users/8b1b4cc85bda4ffe948131a8b7113ada => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:56:13.425228 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e616c-51b1-4a37-b7dd-705b24bc5d7c None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:13.425511 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e616c-51b1-4a37-b7dd-705b24bc5d7c None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:13.425781 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e616c-51b1-4a37-b7dd-705b24bc5d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:13.444085 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-249e616c-51b1-4a37-b7dd-705b24bc5d7c None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:56:13.445282 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-249e616c-51b1-4a37-b7dd-705b24bc5d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:13.503978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1400/2792] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:56:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:56:13.515112 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e13dd3-95a9-4cf4-9ea1-bb6f9b25e8c7 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:13.516629 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-c1e13dd3-95a9-4cf4-9ea1-bb6f9b25e8c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:13.576543 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e13dd3-95a9-4cf4-9ea1-bb6f9b25e8c7 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:13.580204 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f23416-7225-474c-baff-30d5fb38cb4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:13.580802 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f23416-7225-474c-baff-30d5fb38cb4e service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:13.581081 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f23416-7225-474c-baff-30d5fb38cb4e service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:13.581352 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f23416-7225-474c-baff-30d5fb38cb4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:13.590577 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f23416-7225-474c-baff-30d5fb38cb4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:13.592037 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f23416-7225-474c-baff-30d5fb38cb4e service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:13.601170 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1393/2793] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:56:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:56:13.835416 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e616c-51b1-4a37-b7dd-705b24bc5d7c None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:13.836936 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-249e616c-51b1-4a37-b7dd-705b24bc5d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:13.871091 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f23416-7225-474c-baff-30d5fb38cb4e service neutron] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:13.872475 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-e0f23416-7225-474c-baff-30d5fb38cb4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:13.885972 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f23416-7225-474c-baff-30d5fb38cb4e service neutron] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:13.890702 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de91d2d-2649-4b90-9933-b7f2ac379246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:13.891330 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de91d2d-2649-4b90-9933-b7f2ac379246 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:13.891604 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de91d2d-2649-4b90-9933-b7f2ac379246 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:13.891871 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de91d2d-2649-4b90-9933-b7f2ac379246 admin admin] PATH_INFO: `/v3/projects/adca35ce4ab54a7284a8282851c9623e` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:13.893024 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de91d2d-2649-4b90-9933-b7f2ac379246 admin admin] RBAC: Authorizing `identity:delete_project(project_id=adca35ce4ab54a7284a8282851c9623e)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:13.895078 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de91d2d-2649-4b90-9933-b7f2ac379246 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:13.904398 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e616c-51b1-4a37-b7dd-705b24bc5d7c None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:13.908114 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb1d70-88dd-4b31-9720-4f89ccebd0ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:13.908754 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb1d70-88dd-4b31-9720-4f89ccebd0ca service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:13.909032 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb1d70-88dd-4b31-9720-4f89ccebd0ca service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:13.909302 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb1d70-88dd-4b31-9720-4f89ccebd0ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:13.911057 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-01fb1d70-88dd-4b31-9720-4f89ccebd0ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:13.913770 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:56:13.948380 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-4de91d2d-2649-4b90-9933-b7f2ac379246 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adca35ce4ab54a7284a8282851c9623e', 'request_id': 'req-4de91d2d-2649-4b90-9933-b7f2ac379246'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:13.954406 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-4de91d2d-2649-4b90-9933-b7f2ac379246 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adca35ce4ab54a7284a8282851c9623e', 'request_id': 'req-4de91d2d-2649-4b90-9933-b7f2ac379246'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:13.961880 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-4de91d2d-2649-4b90-9933-b7f2ac379246 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adca35ce4ab54a7284a8282851c9623e', 'request_id': 'req-4de91d2d-2649-4b90-9933-b7f2ac379246'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:13.973019 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-4de91d2d-2649-4b90-9933-b7f2ac379246 admin admin] The token cache is being invalidate because project adca35ce4ab54a7284a8282851c9623e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:56:13.973308 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-4de91d2d-2649-4b90-9933-b7f2ac379246 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:13.973969 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-4de91d2d-2649-4b90-9933-b7f2ac379246 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:13.975556 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1394/2794] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:56:13 2022] DELETE /identity/v3/projects/adca35ce4ab54a7284a8282851c9623e => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:56:13.977214 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fb1d70-88dd-4b31-9720-4f89ccebd0ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:13.979316 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fb1d70-88dd-4b31-9720-4f89ccebd0ca service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:13.980975 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de91d2d-2649-4b90-9933-b7f2ac379246 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:13.982269 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-4de91d2d-2649-4b90-9933-b7f2ac379246 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:13.989515 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1401/2795] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:56:13 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:56:14.045769 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de91d2d-2649-4b90-9933-b7f2ac379246 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:14.050362 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fde059-bc04-4877-b79f-37134521b7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:14.050983 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fde059-bc04-4877-b79f-37134521b7c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:14.051681 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fde059-bc04-4877-b79f-37134521b7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:14.051866 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fde059-bc04-4877-b79f-37134521b7c9 admin admin] PATH_INFO: `/v3/users/796d115d08a84bb98ec30d6a2cc7e459` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:14.053022 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fde059-bc04-4877-b79f-37134521b7c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=796d115d08a84bb98ec30d6a2cc7e459)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:14.055179 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fde059-bc04-4877-b79f-37134521b7c9 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:14.103360 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-92fde059-bc04-4877-b79f-37134521b7c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '796d115d08a84bb98ec30d6a2cc7e459'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:14.104075 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-92fde059-bc04-4877-b79f-37134521b7c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '796d115d08a84bb98ec30d6a2cc7e459'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:14.104702 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-92fde059-bc04-4877-b79f-37134521b7c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '796d115d08a84bb98ec30d6a2cc7e459'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:14.111984 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-92fde059-bc04-4877-b79f-37134521b7c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '796d115d08a84bb98ec30d6a2cc7e459'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:14.119670 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-92fde059-bc04-4877-b79f-37134521b7c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '796d115d08a84bb98ec30d6a2cc7e459'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:14.138498 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1395/2796] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:56:13 2022] DELETE /identity/v3/users/796d115d08a84bb98ec30d6a2cc7e459 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:56:14.432798 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb1d70-88dd-4b31-9720-4f89ccebd0ca service cinder] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:14.444177 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb1d70-88dd-4b31-9720-4f89ccebd0ca service cinder] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:14.502739 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=100 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:56:14.513241 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c8b1f-6a6e-4f78-9e8a-9358adc9ef79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:14.513984 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c8b1f-6a6e-4f78-9e8a-9358adc9ef79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:14.514350 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c8b1f-6a6e-4f78-9e8a-9358adc9ef79 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:14.514684 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c8b1f-6a6e-4f78-9e8a-9358adc9ef79 admin admin] PATH_INFO: `/v3/projects/cd576de14bab4393848f2299c21f61f3` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:14.515930 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c8b1f-6a6e-4f78-9e8a-9358adc9ef79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd576de14bab4393848f2299c21f61f3)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:14.517964 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c8b1f-6a6e-4f78-9e8a-9358adc9ef79 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:14.570536 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-ab3c8b1f-6a6e-4f78-9e8a-9358adc9ef79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd576de14bab4393848f2299c21f61f3', 'request_id': 'req-ab3c8b1f-6a6e-4f78-9e8a-9358adc9ef79'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:14.575983 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-ab3c8b1f-6a6e-4f78-9e8a-9358adc9ef79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd576de14bab4393848f2299c21f61f3', 'request_id': 'req-ab3c8b1f-6a6e-4f78-9e8a-9358adc9ef79'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:14.584797 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-ab3c8b1f-6a6e-4f78-9e8a-9358adc9ef79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd576de14bab4393848f2299c21f61f3', 'request_id': 'req-ab3c8b1f-6a6e-4f78-9e8a-9358adc9ef79'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:14.597049 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-ab3c8b1f-6a6e-4f78-9e8a-9358adc9ef79 admin admin] The token cache is being invalidate because project cd576de14bab4393848f2299c21f61f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:56:14.597474 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-ab3c8b1f-6a6e-4f78-9e8a-9358adc9ef79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:14.598272 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-ab3c8b1f-6a6e-4f78-9e8a-9358adc9ef79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:14.600079 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1402/2797] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:56:14 2022] DELETE /identity/v3/projects/cd576de14bab4393848f2299c21f61f3 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:56:14.605117 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fde059-bc04-4877-b79f-37134521b7c9 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:14.606703 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-92fde059-bc04-4877-b79f-37134521b7c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:14.623510 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fde059-bc04-4877-b79f-37134521b7c9 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:14.644443 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:56:14.682119 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ce128-af04-4a19-a593-cb9563bfa454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:14.682742 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ce128-af04-4a19-a593-cb9563bfa454 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:14.683018 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ce128-af04-4a19-a593-cb9563bfa454 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:14.683288 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ce128-af04-4a19-a593-cb9563bfa454 admin admin] PATH_INFO: `/v3/users/544c550d7aec41af9c6bfc41188b3a4d` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:14.684426 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050ce128-af04-4a19-a593-cb9563bfa454 admin admin] RBAC: Authorizing `identity:delete_user(user_id=544c550d7aec41af9c6bfc41188b3a4d)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:14.686319 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050ce128-af04-4a19-a593-cb9563bfa454 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:14.730848 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-050ce128-af04-4a19-a593-cb9563bfa454 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '544c550d7aec41af9c6bfc41188b3a4d'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:14.731519 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-050ce128-af04-4a19-a593-cb9563bfa454 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '544c550d7aec41af9c6bfc41188b3a4d'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:14.732154 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-050ce128-af04-4a19-a593-cb9563bfa454 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '544c550d7aec41af9c6bfc41188b3a4d'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:14.739165 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-050ce128-af04-4a19-a593-cb9563bfa454 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '544c550d7aec41af9c6bfc41188b3a4d'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:14.745923 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-050ce128-af04-4a19-a593-cb9563bfa454 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '544c550d7aec41af9c6bfc41188b3a4d'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:14.763374 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1396/2798] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:56:14 2022] DELETE /identity/v3/users/544c550d7aec41af9c6bfc41188b3a4d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:56:15.025432 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c8b1f-6a6e-4f78-9e8a-9358adc9ef79 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:15.026897 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-ab3c8b1f-6a6e-4f78-9e8a-9358adc9ef79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:15.039586 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c8b1f-6a6e-4f78-9e8a-9358adc9ef79 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:15.098554 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4925c33a-d2bb-4a67-809f-9054fb091927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:15.099174 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925c33a-d2bb-4a67-809f-9054fb091927 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:15.099450 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925c33a-d2bb-4a67-809f-9054fb091927 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:15.099718 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925c33a-d2bb-4a67-809f-9054fb091927 admin admin] PATH_INFO: `/v3/projects/919f80553b1f4995a574c3de3541502b` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:15.100862 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4925c33a-d2bb-4a67-809f-9054fb091927 admin admin] RBAC: Authorizing `identity:delete_project(project_id=919f80553b1f4995a574c3de3541502b)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:15.102825 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4925c33a-d2bb-4a67-809f-9054fb091927 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:15.148880 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-4925c33a-d2bb-4a67-809f-9054fb091927 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '919f80553b1f4995a574c3de3541502b', 'request_id': 'req-4925c33a-d2bb-4a67-809f-9054fb091927'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:15.154584 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-4925c33a-d2bb-4a67-809f-9054fb091927 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '919f80553b1f4995a574c3de3541502b', 'request_id': 'req-4925c33a-d2bb-4a67-809f-9054fb091927'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:15.161281 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-4925c33a-d2bb-4a67-809f-9054fb091927 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '919f80553b1f4995a574c3de3541502b', 'request_id': 'req-4925c33a-d2bb-4a67-809f-9054fb091927'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:15.172306 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-4925c33a-d2bb-4a67-809f-9054fb091927 admin admin] The token cache is being invalidate because project 919f80553b1f4995a574c3de3541502b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:56:15.172589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-4925c33a-d2bb-4a67-809f-9054fb091927 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:15.173205 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-4925c33a-d2bb-4a67-809f-9054fb091927 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:15.174670 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1403/2799] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:56:15 2022] DELETE /identity/v3/projects/919f80553b1f4995a574c3de3541502b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:56:15.181170 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96201d4-b7af-40e9-b109-de562390d37d None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:15.181670 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96201d4-b7af-40e9-b109-de562390d37d None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:15.181786 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96201d4-b7af-40e9-b109-de562390d37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:15.201594 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-e96201d4-b7af-40e9-b109-de562390d37d None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:56:15.202792 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-e96201d4-b7af-40e9-b109-de562390d37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:15.262414 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1397/2800] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:56:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:56:15.271416 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcddc5d0-c0b7-4b85-8000-0e21842742d6 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:15.271703 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcddc5d0-c0b7-4b85-8000-0e21842742d6 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:15.272008 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcddc5d0-c0b7-4b85-8000-0e21842742d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:15.292244 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-fcddc5d0-c0b7-4b85-8000-0e21842742d6 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:56:15.292895 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-fcddc5d0-c0b7-4b85-8000-0e21842742d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:15.301631 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1404/2801] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:56:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:56:15.307120 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96201d4-b7af-40e9-b109-de562390d37d None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:15.317937 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96201d4-b7af-40e9-b109-de562390d37d None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:15.323420 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e82b9b4-da7d-489a-98a7-08e987cfa866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:15.324124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82b9b4-da7d-489a-98a7-08e987cfa866 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:15.324588 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82b9b4-da7d-489a-98a7-08e987cfa866 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:15.324936 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82b9b4-da7d-489a-98a7-08e987cfa866 admin admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:15.325890 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e82b9b4-da7d-489a-98a7-08e987cfa866 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:56:15.326264 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e82b9b4-da7d-489a-98a7-08e987cfa866 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:15.328016 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e82b9b4-da7d-489a-98a7-08e987cfa866 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:15.343978 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1398/2802] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:56:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:56:15.348308 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcddc5d0-c0b7-4b85-8000-0e21842742d6 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:15.354478 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:56:15.358773 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcddc5d0-c0b7-4b85-8000-0e21842742d6 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:15.365325 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae43b92-9c5c-477a-8556-e9366ac6ece3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:15.365931 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae43b92-9c5c-477a-8556-e9366ac6ece3 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:15.366404 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae43b92-9c5c-477a-8556-e9366ac6ece3 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:15.366786 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae43b92-9c5c-477a-8556-e9366ac6ece3 admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:15.367620 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae43b92-9c5c-477a-8556-e9366ac6ece3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:15.370177 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae43b92-9c5c-477a-8556-e9366ac6ece3 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:15.398854 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1405/2803] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:56:15 2022] POST /identity/v3/projects => generated 359 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:56:15.403132 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e82b9b4-da7d-489a-98a7-08e987cfa866 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:15.413565 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e82b9b4-da7d-489a-98a7-08e987cfa866 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:15.447716 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:56:15.478506 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f5d928-3ec5-4208-85fa-ecaf9e97464c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:15.479117 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5d928-3ec5-4208-85fa-ecaf9e97464c admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:15.479403 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5d928-3ec5-4208-85fa-ecaf9e97464c admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:15.479679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5d928-3ec5-4208-85fa-ecaf9e97464c admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:15.480350 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f5d928-3ec5-4208-85fa-ecaf9e97464c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:15.482296 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f5d928-3ec5-4208-85fa-ecaf9e97464c admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:15.508213 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1399/2804] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:56:15 2022] POST /identity/v3/users => generated 280 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:56:15.512556 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae43b92-9c5c-477a-8556-e9366ac6ece3 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:15.523870 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae43b92-9c5c-477a-8556-e9366ac6ece3 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:15.529328 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851e7666-5f9a-4dfd-9252-01b17995985e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:15.530012 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851e7666-5f9a-4dfd-9252-01b17995985e admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:15.530294 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851e7666-5f9a-4dfd-9252-01b17995985e admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:15.530595 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851e7666-5f9a-4dfd-9252-01b17995985e admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:15.531257 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851e7666-5f9a-4dfd-9252-01b17995985e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:15.532967 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851e7666-5f9a-4dfd-9252-01b17995985e admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:15.544424 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1406/2805] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:56:15.548575 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f5d928-3ec5-4208-85fa-ecaf9e97464c admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:15.559942 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f5d928-3ec5-4208-85fa-ecaf9e97464c admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:15.564987 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c62794-9c64-4529-a5f8-f7ee9aed24c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:15.565590 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c62794-9c64-4529-a5f8-f7ee9aed24c4 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:15.565868 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c62794-9c64-4529-a5f8-f7ee9aed24c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:15.566139 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c62794-9c64-4529-a5f8-f7ee9aed24c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:15.566941 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c62794-9c64-4529-a5f8-f7ee9aed24c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:15.568575 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c62794-9c64-4529-a5f8-f7ee9aed24c4 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:15.579306 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1400/2806] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:56:15.583389 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851e7666-5f9a-4dfd-9252-01b17995985e admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:15.595758 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851e7666-5f9a-4dfd-9252-01b17995985e admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:15.600361 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974a5373-9e4b-4f4c-88d1-f96710ca85ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:15.600997 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a5373-9e4b-4f4c-88d1-f96710ca85ee admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:15.601273 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a5373-9e4b-4f4c-88d1-f96710ca85ee admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:15.601542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a5373-9e4b-4f4c-88d1-f96710ca85ee admin admin] PATH_INFO: `/v3/projects/bd54c54b6fb44fec8525b0eded18b6f3/users/d9f2903c29cb4c4a87c9b3745babba96/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:15.616091 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974a5373-9e4b-4f4c-88d1-f96710ca85ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd54c54b6fb44fec8525b0eded18b6f3, user_id=d9f2903c29cb4c4a87c9b3745babba96, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:15.619335 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974a5373-9e4b-4f4c-88d1-f96710ca85ee admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:15.632037 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1407/2807] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:56:15 2022] PUT /identity/v3/projects/bd54c54b6fb44fec8525b0eded18b6f3/users/d9f2903c29cb4c4a87c9b3745babba96/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:56:15.644333 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4155c5-044e-4d23-a8c6-b8954f3cfa42 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:15.644623 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4155c5-044e-4d23-a8c6-b8954f3cfa42 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:15.644895 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4155c5-044e-4d23-a8c6-b8954f3cfa42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:15.673826 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-dd4155c5-044e-4d23-a8c6-b8954f3cfa42 None None] MFA Rules not processed for user `d9f2903c29cb4c4a87c9b3745babba96`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:56:15.675040 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-dd4155c5-044e-4d23-a8c6-b8954f3cfa42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:15.742121 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1401/2808] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:56:15 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:56:15.774942 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974a5373-9e4b-4f4c-88d1-f96710ca85ee admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:15.776651 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-974a5373-9e4b-4f4c-88d1-f96710ca85ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:15.834136 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974a5373-9e4b-4f4c-88d1-f96710ca85ee admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:15.838016 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f704b63-b179-4261-9601-fd8b522d1a01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:15.838632 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f704b63-b179-4261-9601-fd8b522d1a01 service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:15.838912 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f704b63-b179-4261-9601-fd8b522d1a01 service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:15.839191 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f704b63-b179-4261-9601-fd8b522d1a01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:15.848542 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f704b63-b179-4261-9601-fd8b522d1a01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:15.850061 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f704b63-b179-4261-9601-fd8b522d1a01 service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:15.860768 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1408/2809] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:56:15 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:56:15.876926 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4155c5-044e-4d23-a8c6-b8954f3cfa42 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:15.949352 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4155c5-044e-4d23-a8c6-b8954f3cfa42 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:15.954950 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b85d9-d9cf-43f3-b918-0ab075b129d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:15.955591 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b85d9-d9cf-43f3-b918-0ab075b129d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:15.955866 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b85d9-d9cf-43f3-b918-0ab075b129d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:15.956188 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b85d9-d9cf-43f3-b918-0ab075b129d1 admin admin] PATH_INFO: `/v3/users/d9f2903c29cb4c4a87c9b3745babba96` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:15.957323 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133b85d9-d9cf-43f3-b918-0ab075b129d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9f2903c29cb4c4a87c9b3745babba96)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:15.960105 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133b85d9-d9cf-43f3-b918-0ab075b129d1 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:16.004161 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:56:16.011230 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-133b85d9-d9cf-43f3-b918-0ab075b129d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9f2903c29cb4c4a87c9b3745babba96'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:16.012007 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-133b85d9-d9cf-43f3-b918-0ab075b129d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9f2903c29cb4c4a87c9b3745babba96'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:16.012931 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-133b85d9-d9cf-43f3-b918-0ab075b129d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9f2903c29cb4c4a87c9b3745babba96'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:16.019802 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-133b85d9-d9cf-43f3-b918-0ab075b129d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9f2903c29cb4c4a87c9b3745babba96'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:16.027377 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-133b85d9-d9cf-43f3-b918-0ab075b129d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9f2903c29cb4c4a87c9b3745babba96'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:16.046123 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1402/2810] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:56:15 2022] DELETE /identity/v3/users/d9f2903c29cb4c4a87c9b3745babba96 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:56:16.050568 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfda3f2-caf6-40dc-aaee-f7c25dddf8e2 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:16.050908 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfda3f2-caf6-40dc-aaee-f7c25dddf8e2 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:16.051254 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfda3f2-caf6-40dc-aaee-f7c25dddf8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:16.071041 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-dbfda3f2-caf6-40dc-aaee-f7c25dddf8e2 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:56:16.072479 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-dbfda3f2-caf6-40dc-aaee-f7c25dddf8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:16.139746 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1409/2811] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:56:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:56:16.151573 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b85d9-d9cf-43f3-b918-0ab075b129d1 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:16.153107 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-133b85d9-d9cf-43f3-b918-0ab075b129d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:16.221242 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b85d9-d9cf-43f3-b918-0ab075b129d1 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:16.224870 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c69ee4-7de4-4524-bad0-e8119d5752d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:16.225473 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c69ee4-7de4-4524-bad0-e8119d5752d2 service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:16.225750 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c69ee4-7de4-4524-bad0-e8119d5752d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:16.226022 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c69ee4-7de4-4524-bad0-e8119d5752d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:16.235364 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c69ee4-7de4-4524-bad0-e8119d5752d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:16.236822 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c69ee4-7de4-4524-bad0-e8119d5752d2 service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:16.246038 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1403/2812] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:56:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:56:16.515850 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfda3f2-caf6-40dc-aaee-f7c25dddf8e2 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:16.517387 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-dbfda3f2-caf6-40dc-aaee-f7c25dddf8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:16.530207 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfda3f2-caf6-40dc-aaee-f7c25dddf8e2 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:16.535151 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e2af95-279f-40b6-b7fc-3a5082b5052e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:16.535778 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2af95-279f-40b6-b7fc-3a5082b5052e admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:16.536079 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2af95-279f-40b6-b7fc-3a5082b5052e admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:16.536348 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2af95-279f-40b6-b7fc-3a5082b5052e admin admin] PATH_INFO: `/v3/projects/bd54c54b6fb44fec8525b0eded18b6f3` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:16.537484 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e2af95-279f-40b6-b7fc-3a5082b5052e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd54c54b6fb44fec8525b0eded18b6f3)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:16.539413 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e2af95-279f-40b6-b7fc-3a5082b5052e admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:16.583346 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:56:16.586428 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-55e2af95-279f-40b6-b7fc-3a5082b5052e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd54c54b6fb44fec8525b0eded18b6f3', 'request_id': 'req-55e2af95-279f-40b6-b7fc-3a5082b5052e'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:16.593206 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-55e2af95-279f-40b6-b7fc-3a5082b5052e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd54c54b6fb44fec8525b0eded18b6f3', 'request_id': 'req-55e2af95-279f-40b6-b7fc-3a5082b5052e'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:16.600378 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-55e2af95-279f-40b6-b7fc-3a5082b5052e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd54c54b6fb44fec8525b0eded18b6f3', 'request_id': 'req-55e2af95-279f-40b6-b7fc-3a5082b5052e'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:16.611823 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-55e2af95-279f-40b6-b7fc-3a5082b5052e admin admin] The token cache is being invalidate because project bd54c54b6fb44fec8525b0eded18b6f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:56:16.612203 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-55e2af95-279f-40b6-b7fc-3a5082b5052e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:16.612847 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-55e2af95-279f-40b6-b7fc-3a5082b5052e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:16.614328 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1410/2813] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:56:16 2022] DELETE /identity/v3/projects/bd54c54b6fb44fec8525b0eded18b6f3 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:56:16.619705 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561646e0-0b75-43b6-ac86-8e1d5ec631ee None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:16.620102 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561646e0-0b75-43b6-ac86-8e1d5ec631ee None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:16.620377 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561646e0-0b75-43b6-ac86-8e1d5ec631ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:16.647142 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-561646e0-0b75-43b6-ac86-8e1d5ec631ee None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:56:16.648362 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-561646e0-0b75-43b6-ac86-8e1d5ec631ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:16.715846 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1404/2814] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:56:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:56:16.724026 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0d092-1b4c-46b2-b542-2c165cf8cc8a None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:16.724321 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0d092-1b4c-46b2-b542-2c165cf8cc8a None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:16.724596 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0d092-1b4c-46b2-b542-2c165cf8cc8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:16.746524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-9df0d092-1b4c-46b2-b542-2c165cf8cc8a None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:56:16.747737 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-9df0d092-1b4c-46b2-b542-2c165cf8cc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:16.756492 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1411/2815] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:56:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:56:16.764570 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561646e0-0b75-43b6-ac86-8e1d5ec631ee None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:16.771601 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561646e0-0b75-43b6-ac86-8e1d5ec631ee None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:16.780959 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a543161-1af7-4cf0-988c-e6ce7415d215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:16.781664 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a543161-1af7-4cf0-988c-e6ce7415d215 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:16.781946 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a543161-1af7-4cf0-988c-e6ce7415d215 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:16.782217 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a543161-1af7-4cf0-988c-e6ce7415d215 admin admin] PATH_INFO: `/v3/domains` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:16.782940 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a543161-1af7-4cf0-988c-e6ce7415d215 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85834) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:56:16.783250 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a543161-1af7-4cf0-988c-e6ce7415d215 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:16.784906 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a543161-1af7-4cf0-988c-e6ce7415d215 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:16.797965 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1405/2816] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:56:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:56:16.802358 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df0d092-1b4c-46b2-b542-2c165cf8cc8a None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:16.812954 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df0d092-1b4c-46b2-b542-2c165cf8cc8a None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:16.818047 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf22ae-84df-4d49-89fb-627b30b0ff65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:16.818645 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf22ae-84df-4d49-89fb-627b30b0ff65 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:16.818923 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf22ae-84df-4d49-89fb-627b30b0ff65 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:16.819192 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf22ae-84df-4d49-89fb-627b30b0ff65 admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:16.819815 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf22ae-84df-4d49-89fb-627b30b0ff65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:16.822417 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf22ae-84df-4d49-89fb-627b30b0ff65 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:16.858198 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1412/2817] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:56:16 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:56:16.862654 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a543161-1af7-4cf0-988c-e6ce7415d215 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:16.874454 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a543161-1af7-4cf0-988c-e6ce7415d215 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:16.925776 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:56:16.941249 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efe443-9f27-431f-943d-b0d153b8a87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:16.941791 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efe443-9f27-431f-943d-b0d153b8a87c admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:16.942125 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efe443-9f27-431f-943d-b0d153b8a87c admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:16.942469 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efe443-9f27-431f-943d-b0d153b8a87c admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:16.943109 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15efe443-9f27-431f-943d-b0d153b8a87c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:16.945081 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15efe443-9f27-431f-943d-b0d153b8a87c admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:16.971317 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1406/2818] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:56:16 2022] POST /identity/v3/users => generated 288 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:56:16.972366 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf22ae-84df-4d49-89fb-627b30b0ff65 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:16.983082 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf22ae-84df-4d49-89fb-627b30b0ff65 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:16.987769 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166886ed-fd77-4c77-9e46-aa55deab6400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:16.988380 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166886ed-fd77-4c77-9e46-aa55deab6400 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:16.988656 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166886ed-fd77-4c77-9e46-aa55deab6400 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:16.988921 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166886ed-fd77-4c77-9e46-aa55deab6400 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:16.989556 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166886ed-fd77-4c77-9e46-aa55deab6400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:16.991150 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166886ed-fd77-4c77-9e46-aa55deab6400 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:17.001679 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1413/2819] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:56:17.005862 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efe443-9f27-431f-943d-b0d153b8a87c admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:17.016231 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efe443-9f27-431f-943d-b0d153b8a87c admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:17.020911 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba6a95d-2652-4490-90a8-ad3ac21ce812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:17.021507 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6a95d-2652-4490-90a8-ad3ac21ce812 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:17.021780 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6a95d-2652-4490-90a8-ad3ac21ce812 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:17.022047 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6a95d-2652-4490-90a8-ad3ac21ce812 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:17.022679 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba6a95d-2652-4490-90a8-ad3ac21ce812 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:17.024288 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba6a95d-2652-4490-90a8-ad3ac21ce812 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:17.036108 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1407/2820] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:56:17.040853 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166886ed-fd77-4c77-9e46-aa55deab6400 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:17.056414 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166886ed-fd77-4c77-9e46-aa55deab6400 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:17.062114 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c7573-6e3d-452e-bb7f-845457ccd792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:17.063554 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c7573-6e3d-452e-bb7f-845457ccd792 admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:17.063554 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c7573-6e3d-452e-bb7f-845457ccd792 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:17.063554 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c7573-6e3d-452e-bb7f-845457ccd792 admin admin] PATH_INFO: `/v3/projects/430c456402984ab2ad3e75e7213719f9/users/0ec581e93c434c1195649618ec8f4e2f/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:17.079157 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c7573-6e3d-452e-bb7f-845457ccd792 admin admin] RBAC: Authorizing `identity:create_grant(project_id=430c456402984ab2ad3e75e7213719f9, user_id=0ec581e93c434c1195649618ec8f4e2f, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:17.081907 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c7573-6e3d-452e-bb7f-845457ccd792 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:17.095315 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1414/2821] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:56:17 2022] PUT /identity/v3/projects/430c456402984ab2ad3e75e7213719f9/users/0ec581e93c434c1195649618ec8f4e2f/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:56:17.106062 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851eda5a-7383-4a00-aa22-8ebb334ffcff None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:17.106350 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851eda5a-7383-4a00-aa22-8ebb334ffcff None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:17.106624 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851eda5a-7383-4a00-aa22-8ebb334ffcff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:17.134859 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-851eda5a-7383-4a00-aa22-8ebb334ffcff None None] MFA Rules not processed for user `0ec581e93c434c1195649618ec8f4e2f`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:56:17.136095 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-851eda5a-7383-4a00-aa22-8ebb334ffcff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:17.201906 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1408/2822] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:56:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:56:17.206417 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c7573-6e3d-452e-bb7f-845457ccd792 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:17.268187 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c7573-6e3d-452e-bb7f-845457ccd792 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:17.272818 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c6e681-619c-410d-80b9-41133a364017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:17.273423 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6e681-619c-410d-80b9-41133a364017 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:17.273701 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6e681-619c-410d-80b9-41133a364017 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:17.273979 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6e681-619c-410d-80b9-41133a364017 admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:17.274621 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c6e681-619c-410d-80b9-41133a364017 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:17.276568 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c6e681-619c-410d-80b9-41133a364017 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:17.302674 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1415/2823] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:56:17 2022] POST /identity/v3/projects => generated 375 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:56:17.307292 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851eda5a-7383-4a00-aa22-8ebb334ffcff None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:17.317825 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851eda5a-7383-4a00-aa22-8ebb334ffcff None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:17.374495 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e90a6f1-2ea4-466e-a7fa-202e08701f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:17.375102 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90a6f1-2ea4-466e-a7fa-202e08701f79 admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:17.375379 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90a6f1-2ea4-466e-a7fa-202e08701f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:17.375653 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90a6f1-2ea4-466e-a7fa-202e08701f79 admin admin] PATH_INFO: `/v3/users` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:17.376322 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e90a6f1-2ea4-466e-a7fa-202e08701f79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:17.378237 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e90a6f1-2ea4-466e-a7fa-202e08701f79 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:17.398474 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1409/2824] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:56:17 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:56:17.403029 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c6e681-619c-410d-80b9-41133a364017 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:17.413311 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c6e681-619c-410d-80b9-41133a364017 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:17.418806 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7edfb8-09a4-4459-a3bb-91cc5ac35d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:17.419394 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7edfb8-09a4-4459-a3bb-91cc5ac35d69 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:17.419677 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7edfb8-09a4-4459-a3bb-91cc5ac35d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:17.420005 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7edfb8-09a4-4459-a3bb-91cc5ac35d69 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:17.420646 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7edfb8-09a4-4459-a3bb-91cc5ac35d69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:17.422238 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7edfb8-09a4-4459-a3bb-91cc5ac35d69 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:17.432656 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1416/2825] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:56:17.436986 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e90a6f1-2ea4-466e-a7fa-202e08701f79 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:17.447124 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e90a6f1-2ea4-466e-a7fa-202e08701f79 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:17.451678 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d5c34-7962-4327-bec4-9425c237873b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:17.452451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d5c34-7962-4327-bec4-9425c237873b admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:17.452687 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d5c34-7962-4327-bec4-9425c237873b admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:17.452961 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d5c34-7962-4327-bec4-9425c237873b admin admin] PATH_INFO: `/v3/projects/67b7110eb2f64fb5a807205e20d8691e/users/d3ca50c5b48b49ef9cddbb626d5a68ba/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:17.468093 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d5c34-7962-4327-bec4-9425c237873b admin admin] RBAC: Authorizing `identity:create_grant(project_id=67b7110eb2f64fb5a807205e20d8691e, user_id=d3ca50c5b48b49ef9cddbb626d5a68ba, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:17.470736 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d5c34-7962-4327-bec4-9425c237873b admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:17.483420 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1410/2826] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:56:17 2022] PUT /identity/v3/projects/67b7110eb2f64fb5a807205e20d8691e/users/d3ca50c5b48b49ef9cddbb626d5a68ba/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:56:17.488318 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7edfb8-09a4-4459-a3bb-91cc5ac35d69 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:17.498472 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7edfb8-09a4-4459-a3bb-91cc5ac35d69 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:17.538972 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:56:17.556128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c2426-3dbc-4c2c-bf2b-614339d144c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:17.556733 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c2426-3dbc-4c2c-bf2b-614339d144c7 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:17.557019 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c2426-3dbc-4c2c-bf2b-614339d144c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:17.557288 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c2426-3dbc-4c2c-bf2b-614339d144c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:17.557932 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6c2426-3dbc-4c2c-bf2b-614339d144c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:17.559535 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6c2426-3dbc-4c2c-bf2b-614339d144c7 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:17.569257 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1417/2827] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:56:17.573436 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d5c34-7962-4327-bec4-9425c237873b admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:17.583562 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d5c34-7962-4327-bec4-9425c237873b admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:17.588644 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c4723-502a-4493-9d13-6aab90dbe8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:17.589284 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c4723-502a-4493-9d13-6aab90dbe8ec admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:17.589564 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c4723-502a-4493-9d13-6aab90dbe8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:17.589836 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c4723-502a-4493-9d13-6aab90dbe8ec admin admin] PATH_INFO: `/v3/projects/67b7110eb2f64fb5a807205e20d8691e/users/d3ca50c5b48b49ef9cddbb626d5a68ba/roles/33bd60d3b0e94dcbbfe97e87f91a8d78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:17.591744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476c4723-502a-4493-9d13-6aab90dbe8ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=67b7110eb2f64fb5a807205e20d8691e, user_id=d3ca50c5b48b49ef9cddbb626d5a68ba, role_id=33bd60d3b0e94dcbbfe97e87f91a8d78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:17.594389 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476c4723-502a-4493-9d13-6aab90dbe8ec admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:17.604972 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1411/2828] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:56:17 2022] PUT /identity/v3/projects/67b7110eb2f64fb5a807205e20d8691e/users/d3ca50c5b48b49ef9cddbb626d5a68ba/roles/33bd60d3b0e94dcbbfe97e87f91a8d78 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:56:17.669315 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d73db-29f4-40db-87e2-2825d47fb27f None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:17.669622 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d73db-29f4-40db-87e2-2825d47fb27f None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:17.669900 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d73db-29f4-40db-87e2-2825d47fb27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:17.697462 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-892d73db-29f4-40db-87e2-2825d47fb27f None None] MFA Rules not processed for user `d3ca50c5b48b49ef9cddbb626d5a68ba`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:56:17.698774 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-892d73db-29f4-40db-87e2-2825d47fb27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:17.769071 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1418/2829] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:56:17 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:56:17.780235 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c4723-502a-4493-9d13-6aab90dbe8ec admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:17.781913 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-476c4723-502a-4493-9d13-6aab90dbe8ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:17.792639 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:56:17.845561 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c4723-502a-4493-9d13-6aab90dbe8ec admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:17.849294 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2006f6ea-01ea-4e29-a5bc-076fa7eb1f87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:17.849931 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006f6ea-01ea-4e29-a5bc-076fa7eb1f87 service cinder] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:17.850212 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006f6ea-01ea-4e29-a5bc-076fa7eb1f87 service cinder] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:17.850618 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006f6ea-01ea-4e29-a5bc-076fa7eb1f87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:17.916383 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2006f6ea-01ea-4e29-a5bc-076fa7eb1f87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:17.920509 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d73db-29f4-40db-87e2-2825d47fb27f None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:17.920818 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2006f6ea-01ea-4e29-a5bc-076fa7eb1f87 service cinder] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:17.921888 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-892d73db-29f4-40db-87e2-2825d47fb27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:17.929903 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1412/2830] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:56:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:56:17.989420 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d73db-29f4-40db-87e2-2825d47fb27f None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:17.994111 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94309629-6f6d-4aa2-ba04-23809013de43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:17.994792 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94309629-6f6d-4aa2-ba04-23809013de43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:17.995097 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94309629-6f6d-4aa2-ba04-23809013de43 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:17.995371 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94309629-6f6d-4aa2-ba04-23809013de43 admin admin] PATH_INFO: `/v3/users/919c6627d8c14afdaa60920359f617f1` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:17.996558 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94309629-6f6d-4aa2-ba04-23809013de43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=919c6627d8c14afdaa60920359f617f1)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:17.998598 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94309629-6f6d-4aa2-ba04-23809013de43 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:18.050286 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-94309629-6f6d-4aa2-ba04-23809013de43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '919c6627d8c14afdaa60920359f617f1'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:18.051203 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-94309629-6f6d-4aa2-ba04-23809013de43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '919c6627d8c14afdaa60920359f617f1'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:18.066398 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-94309629-6f6d-4aa2-ba04-23809013de43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '919c6627d8c14afdaa60920359f617f1'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:18.070246 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:56:18.077258 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-94309629-6f6d-4aa2-ba04-23809013de43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '919c6627d8c14afdaa60920359f617f1'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:18.078094 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-94309629-6f6d-4aa2-ba04-23809013de43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '919c6627d8c14afdaa60920359f617f1'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:18.098599 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1419/2831] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:56:17 2022] DELETE /identity/v3/users/919c6627d8c14afdaa60920359f617f1 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:56:18.104059 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a89a5c-f333-41ba-8779-1326363a8128 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:18.104390 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a89a5c-f333-41ba-8779-1326363a8128 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:18.104705 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a89a5c-f333-41ba-8779-1326363a8128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:18.125963 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-56a89a5c-f333-41ba-8779-1326363a8128 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:56:18.127261 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-56a89a5c-f333-41ba-8779-1326363a8128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:18.200216 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1413/2832] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:56:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:56:18.210194 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94309629-6f6d-4aa2-ba04-23809013de43 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:18.212141 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-94309629-6f6d-4aa2-ba04-23809013de43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:18.283412 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94309629-6f6d-4aa2-ba04-23809013de43 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:18.287217 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce514834-60a4-48a2-b5bc-197b6e4dfce2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:18.287929 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce514834-60a4-48a2-b5bc-197b6e4dfce2 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:18.288336 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce514834-60a4-48a2-b5bc-197b6e4dfce2 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:18.288602 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce514834-60a4-48a2-b5bc-197b6e4dfce2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:18.299661 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce514834-60a4-48a2-b5bc-197b6e4dfce2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:18.301110 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce514834-60a4-48a2-b5bc-197b6e4dfce2 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:18.311328 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1420/2833] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:56:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:56:18.575396 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a89a5c-f333-41ba-8779-1326363a8128 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:18.576980 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-56a89a5c-f333-41ba-8779-1326363a8128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:18.592819 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a89a5c-f333-41ba-8779-1326363a8128 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:18.597436 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c1ff2-8cd4-4c72-92dd-761f58365406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:18.598066 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c1ff2-8cd4-4c72-92dd-761f58365406 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:18.598347 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c1ff2-8cd4-4c72-92dd-761f58365406 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:18.598616 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c1ff2-8cd4-4c72-92dd-761f58365406 admin admin] PATH_INFO: `/v3/projects/8a55e99004364307b359d747336649fd` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:18.599744 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016c1ff2-8cd4-4c72-92dd-761f58365406 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a55e99004364307b359d747336649fd)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:18.601682 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016c1ff2-8cd4-4c72-92dd-761f58365406 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:18.649030 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-016c1ff2-8cd4-4c72-92dd-761f58365406 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a55e99004364307b359d747336649fd', 'request_id': 'req-016c1ff2-8cd4-4c72-92dd-761f58365406'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:18.654418 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-016c1ff2-8cd4-4c72-92dd-761f58365406 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a55e99004364307b359d747336649fd', 'request_id': 'req-016c1ff2-8cd4-4c72-92dd-761f58365406'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:18.656077 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:56:18.662012 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-016c1ff2-8cd4-4c72-92dd-761f58365406 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a55e99004364307b359d747336649fd', 'request_id': 'req-016c1ff2-8cd4-4c72-92dd-761f58365406'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:18.674141 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-016c1ff2-8cd4-4c72-92dd-761f58365406 admin admin] The token cache is being invalidate because project 8a55e99004364307b359d747336649fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:56:18.674433 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-016c1ff2-8cd4-4c72-92dd-761f58365406 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:18.675051 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-016c1ff2-8cd4-4c72-92dd-761f58365406 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:18.676580 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1414/2834] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:56:18 2022] DELETE /identity/v3/projects/8a55e99004364307b359d747336649fd => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:56:18.901891 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce514834-60a4-48a2-b5bc-197b6e4dfce2 service neutron] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:18.903334 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-ce514834-60a4-48a2-b5bc-197b6e4dfce2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:18.917575 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce514834-60a4-48a2-b5bc-197b6e4dfce2 service neutron] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:18.976231 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c1d8d-1f5a-43af-b90c-4ad5b8ab9991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:18.976858 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c1d8d-1f5a-43af-b90c-4ad5b8ab9991 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:18.977137 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c1d8d-1f5a-43af-b90c-4ad5b8ab9991 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:18.978041 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c1d8d-1f5a-43af-b90c-4ad5b8ab9991 admin admin] PATH_INFO: `/v3/users/5bb3ba4bfb554e719c8f440698a17c4f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:18.979219 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8c1d8d-1f5a-43af-b90c-4ad5b8ab9991 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bb3ba4bfb554e719c8f440698a17c4f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:18.981181 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8c1d8d-1f5a-43af-b90c-4ad5b8ab9991 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:19.030494 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-0d8c1d8d-1f5a-43af-b90c-4ad5b8ab9991 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bb3ba4bfb554e719c8f440698a17c4f'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:19.031148 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-0d8c1d8d-1f5a-43af-b90c-4ad5b8ab9991 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bb3ba4bfb554e719c8f440698a17c4f'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:19.044607 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-0d8c1d8d-1f5a-43af-b90c-4ad5b8ab9991 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bb3ba4bfb554e719c8f440698a17c4f'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:19.053397 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-0d8c1d8d-1f5a-43af-b90c-4ad5b8ab9991 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bb3ba4bfb554e719c8f440698a17c4f'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:19.054148 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-0d8c1d8d-1f5a-43af-b90c-4ad5b8ab9991 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bb3ba4bfb554e719c8f440698a17c4f'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:19.067136 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1421/2835] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:56:18 2022] DELETE /identity/v3/users/5bb3ba4bfb554e719c8f440698a17c4f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:56:19.071985 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4f608-8364-4849-a72e-2ff12e843548 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:19.072291 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4f608-8364-4849-a72e-2ff12e843548 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:19.072567 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4f608-8364-4849-a72e-2ff12e843548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:19.092477 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-d0e4f608-8364-4849-a72e-2ff12e843548 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:56:19.093713 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-d0e4f608-8364-4849-a72e-2ff12e843548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:19.151305 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1415/2836] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:56:19.161161 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c1d8d-1f5a-43af-b90c-4ad5b8ab9991 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:19.162807 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-0d8c1d8d-1f5a-43af-b90c-4ad5b8ab9991 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:19.173573 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:56:19.222105 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c1d8d-1f5a-43af-b90c-4ad5b8ab9991 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:19.225922 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631d4af5-dec1-4679-b956-570950003b27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:19.226525 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d4af5-dec1-4679-b956-570950003b27 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:19.226803 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d4af5-dec1-4679-b956-570950003b27 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:19.227073 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d4af5-dec1-4679-b956-570950003b27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:19.236733 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631d4af5-dec1-4679-b956-570950003b27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:19.238168 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631d4af5-dec1-4679-b956-570950003b27 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:19.247558 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1422/2837] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:56:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:56:19.799861 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e4f608-8364-4849-a72e-2ff12e843548 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:19.801404 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-d0e4f608-8364-4849-a72e-2ff12e843548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:19.813756 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e4f608-8364-4849-a72e-2ff12e843548 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:19.818364 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb94f1b6-8c1c-4d0e-8024-cd16a781dde3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:19.818996 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb94f1b6-8c1c-4d0e-8024-cd16a781dde3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:19.819269 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb94f1b6-8c1c-4d0e-8024-cd16a781dde3 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:19.819537 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb94f1b6-8c1c-4d0e-8024-cd16a781dde3 admin admin] PATH_INFO: `/v3/projects/66a8f78a096c4f72a806aae2ea820189` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:19.820662 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb94f1b6-8c1c-4d0e-8024-cd16a781dde3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66a8f78a096c4f72a806aae2ea820189)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:19.823095 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb94f1b6-8c1c-4d0e-8024-cd16a781dde3 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:19.869222 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-cb94f1b6-8c1c-4d0e-8024-cd16a781dde3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66a8f78a096c4f72a806aae2ea820189', 'request_id': 'req-cb94f1b6-8c1c-4d0e-8024-cd16a781dde3'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:19.876435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-cb94f1b6-8c1c-4d0e-8024-cd16a781dde3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66a8f78a096c4f72a806aae2ea820189', 'request_id': 'req-cb94f1b6-8c1c-4d0e-8024-cd16a781dde3'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:19.885501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-cb94f1b6-8c1c-4d0e-8024-cd16a781dde3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66a8f78a096c4f72a806aae2ea820189', 'request_id': 'req-cb94f1b6-8c1c-4d0e-8024-cd16a781dde3'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:19.896770 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-cb94f1b6-8c1c-4d0e-8024-cd16a781dde3 admin admin] The token cache is being invalidate because project 66a8f78a096c4f72a806aae2ea820189 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:56:19.897059 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-cb94f1b6-8c1c-4d0e-8024-cd16a781dde3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:19.897684 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-cb94f1b6-8c1c-4d0e-8024-cd16a781dde3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:19.899154 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1416/2838] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:56:19 2022] DELETE /identity/v3/projects/66a8f78a096c4f72a806aae2ea820189 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:56:19.903280 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631d4af5-dec1-4679-b956-570950003b27 service neutron] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:19.912865 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631d4af5-dec1-4679-b956-570950003b27 service neutron] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:19.968555 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d922be6c-fcc0-41e2-bf0f-2d47c4a2e37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:19.969104 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922be6c-fcc0-41e2-bf0f-2d47c4a2e37d admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:19.969383 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922be6c-fcc0-41e2-bf0f-2d47c4a2e37d admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:19.969656 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922be6c-fcc0-41e2-bf0f-2d47c4a2e37d admin admin] PATH_INFO: `/v3/users/1b2bcb5e8ad04fbb8af4364950fe6bcd` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:19.970769 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d922be6c-fcc0-41e2-bf0f-2d47c4a2e37d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b2bcb5e8ad04fbb8af4364950fe6bcd)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:19.972735 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d922be6c-fcc0-41e2-bf0f-2d47c4a2e37d admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:20.018501 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-d922be6c-fcc0-41e2-bf0f-2d47c4a2e37d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b2bcb5e8ad04fbb8af4364950fe6bcd'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:20.019224 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-d922be6c-fcc0-41e2-bf0f-2d47c4a2e37d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b2bcb5e8ad04fbb8af4364950fe6bcd'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:20.030908 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-d922be6c-fcc0-41e2-bf0f-2d47c4a2e37d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b2bcb5e8ad04fbb8af4364950fe6bcd'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:20.037968 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-d922be6c-fcc0-41e2-bf0f-2d47c4a2e37d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b2bcb5e8ad04fbb8af4364950fe6bcd'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:20.038581 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-d922be6c-fcc0-41e2-bf0f-2d47c4a2e37d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b2bcb5e8ad04fbb8af4364950fe6bcd'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:20.051532 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1423/2839] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:56:19 2022] DELETE /identity/v3/users/1b2bcb5e8ad04fbb8af4364950fe6bcd => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:56:20.321284 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb94f1b6-8c1c-4d0e-8024-cd16a781dde3 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:20.322778 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-cb94f1b6-8c1c-4d0e-8024-cd16a781dde3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:20.337922 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb94f1b6-8c1c-4d0e-8024-cd16a781dde3 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:20.395021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b8bb87-654c-4c44-ae23-013a1c0a680a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:20.395646 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b8bb87-654c-4c44-ae23-013a1c0a680a admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:20.395927 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b8bb87-654c-4c44-ae23-013a1c0a680a admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:20.396232 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b8bb87-654c-4c44-ae23-013a1c0a680a admin admin] PATH_INFO: `/v3/projects/fa951d5a9361411e8cc24bdaac5d70a5` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:20.397335 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b8bb87-654c-4c44-ae23-013a1c0a680a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa951d5a9361411e8cc24bdaac5d70a5)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:20.399728 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b8bb87-654c-4c44-ae23-013a1c0a680a admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:20.410660 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:56:20.446559 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-60b8bb87-654c-4c44-ae23-013a1c0a680a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa951d5a9361411e8cc24bdaac5d70a5', 'request_id': 'req-60b8bb87-654c-4c44-ae23-013a1c0a680a'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:20.452007 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-60b8bb87-654c-4c44-ae23-013a1c0a680a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa951d5a9361411e8cc24bdaac5d70a5', 'request_id': 'req-60b8bb87-654c-4c44-ae23-013a1c0a680a'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:20.459394 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-60b8bb87-654c-4c44-ae23-013a1c0a680a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa951d5a9361411e8cc24bdaac5d70a5', 'request_id': 'req-60b8bb87-654c-4c44-ae23-013a1c0a680a'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:20.470878 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-60b8bb87-654c-4c44-ae23-013a1c0a680a admin admin] The token cache is being invalidate because project fa951d5a9361411e8cc24bdaac5d70a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:56:20.471163 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-60b8bb87-654c-4c44-ae23-013a1c0a680a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:20.471801 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-60b8bb87-654c-4c44-ae23-013a1c0a680a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:20.473353 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1417/2840] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:56:20 2022] DELETE /identity/v3/projects/fa951d5a9361411e8cc24bdaac5d70a5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:56:20.480348 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101933c-5779-40bc-af14-2b4ff71a9897 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:20.480640 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101933c-5779-40bc-af14-2b4ff71a9897 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:20.480918 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101933c-5779-40bc-af14-2b4ff71a9897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:20.502893 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-b101933c-5779-40bc-af14-2b4ff71a9897 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:56:20.504144 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-b101933c-5779-40bc-af14-2b4ff71a9897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:20.514792 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b8bb87-654c-4c44-ae23-013a1c0a680a admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:20.516222 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-60b8bb87-654c-4c44-ae23-013a1c0a680a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:20.529451 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:56:20.568080 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1424/2841] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:56:20.576070 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240521c-168b-4b7c-ae6f-611712fbd32c None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:20.576451 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240521c-168b-4b7c-ae6f-611712fbd32c None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:20.576722 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240521c-168b-4b7c-ae6f-611712fbd32c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:20.581741 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b8bb87-654c-4c44-ae23-013a1c0a680a admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:20.586452 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843c139e-de18-4d62-bb10-3607c468a38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:20.587615 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c139e-de18-4d62-bb10-3607c468a38c admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:20.587894 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c139e-de18-4d62-bb10-3607c468a38c admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:20.588193 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c139e-de18-4d62-bb10-3607c468a38c admin admin] PATH_INFO: `/v3/users/0ec581e93c434c1195649618ec8f4e2f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:20.589309 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843c139e-de18-4d62-bb10-3607c468a38c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ec581e93c434c1195649618ec8f4e2f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:20.591251 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843c139e-de18-4d62-bb10-3607c468a38c admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:20.597893 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-c240521c-168b-4b7c-ae6f-611712fbd32c None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:56:20.599158 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-c240521c-168b-4b7c-ae6f-611712fbd32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:20.608176 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1425/2842] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:56:20.611772 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240521c-168b-4b7c-ae6f-611712fbd32c None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:20.621601 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240521c-168b-4b7c-ae6f-611712fbd32c None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:20.632913 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda234c1-fbb5-42f2-b38d-21541a3c022e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:20.633524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda234c1-fbb5-42f2-b38d-21541a3c022e admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:20.634116 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda234c1-fbb5-42f2-b38d-21541a3c022e admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:20.634394 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda234c1-fbb5-42f2-b38d-21541a3c022e admin admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:20.635044 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda234c1-fbb5-42f2-b38d-21541a3c022e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:56:20.635379 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda234c1-fbb5-42f2-b38d-21541a3c022e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:20.637045 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda234c1-fbb5-42f2-b38d-21541a3c022e admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:20.646676 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-843c139e-de18-4d62-bb10-3607c468a38c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ec581e93c434c1195649618ec8f4e2f'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:20.647350 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-843c139e-de18-4d62-bb10-3607c468a38c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ec581e93c434c1195649618ec8f4e2f'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:20.648000 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-843c139e-de18-4d62-bb10-3607c468a38c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ec581e93c434c1195649618ec8f4e2f'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:20.652327 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1426/2843] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:56:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:56:20.653524 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda234c1-fbb5-42f2-b38d-21541a3c022e admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:20.655158 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-bda234c1-fbb5-42f2-b38d-21541a3c022e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:20.659718 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-843c139e-de18-4d62-bb10-3607c468a38c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ec581e93c434c1195649618ec8f4e2f'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:20.668070 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda234c1-fbb5-42f2-b38d-21541a3c022e admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:20.668499 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-843c139e-de18-4d62-bb10-3607c468a38c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ec581e93c434c1195649618ec8f4e2f'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:20.673012 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2127473-d457-46f5-9aec-4d5c2a108b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:20.673589 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2127473-d457-46f5-9aec-4d5c2a108b1f admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:20.673868 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2127473-d457-46f5-9aec-4d5c2a108b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:20.674137 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2127473-d457-46f5-9aec-4d5c2a108b1f admin admin] PATH_INFO: `/v3/projects` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:20.674745 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2127473-d457-46f5-9aec-4d5c2a108b1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:20.676668 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2127473-d457-46f5-9aec-4d5c2a108b1f admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:20.685847 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1418/2844] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:56:20 2022] DELETE /identity/v3/users/0ec581e93c434c1195649618ec8f4e2f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:56:20.690399 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c285e9-fb43-48a7-bcf0-a999ec17c3e7 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:20.690685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c285e9-fb43-48a7-bcf0-a999ec17c3e7 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:20.690966 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c285e9-fb43-48a7-bcf0-a999ec17c3e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:20.706216 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1427/2845] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:56:20 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:56:20.707224 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2127473-d457-46f5-9aec-4d5c2a108b1f admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:20.710954 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-48c285e9-fb43-48a7-bcf0-a999ec17c3e7 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:56:20.712702 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-48c285e9-fb43-48a7-bcf0-a999ec17c3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:20.725296 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:56:20.770624 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2127473-d457-46f5-9aec-4d5c2a108b1f admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:20.772509 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1419/2846] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:56:20.776262 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba2fe9-f23c-4f42-b76b-e7871c962f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:20.777170 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba2fe9-f23c-4f42-b76b-e7871c962f00 admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:20.777474 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba2fe9-f23c-4f42-b76b-e7871c962f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:20.777751 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba2fe9-f23c-4f42-b76b-e7871c962f00 admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:20.778383 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ba2fe9-f23c-4f42-b76b-e7871c962f00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:20.780340 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ba2fe9-f23c-4f42-b76b-e7871c962f00 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:20.782773 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c285e9-fb43-48a7-bcf0-a999ec17c3e7 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:20.784150 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-48c285e9-fb43-48a7-bcf0-a999ec17c3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:20.805502 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1428/2847] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:56:20 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:56:20.809494 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba2fe9-f23c-4f42-b76b-e7871c962f00 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:20.819214 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba2fe9-f23c-4f42-b76b-e7871c962f00 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:20.824398 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1742d-d0d9-454e-b645-c669ea275af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:20.824978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1742d-d0d9-454e-b645-c669ea275af4 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:20.825259 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1742d-d0d9-454e-b645-c669ea275af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:20.825527 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1742d-d0d9-454e-b645-c669ea275af4 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:20.826163 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c1742d-d0d9-454e-b645-c669ea275af4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:20.827741 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c1742d-d0d9-454e-b645-c669ea275af4 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:20.838854 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1429/2848] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 23:56:20.842728 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1742d-d0d9-454e-b645-c669ea275af4 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:20.846882 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c285e9-fb43-48a7-bcf0-a999ec17c3e7 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:20.851936 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccbc5ae-72a0-4ed4-a042-3225427555fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:20.852390 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1742d-d0d9-454e-b645-c669ea275af4 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:20.852591 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccbc5ae-72a0-4ed4-a042-3225427555fb service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:20.852868 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccbc5ae-72a0-4ed4-a042-3225427555fb service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:20.853142 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccbc5ae-72a0-4ed4-a042-3225427555fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:20.857107 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f6bc42-e13a-43ea-b635-7756a1c79a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:20.857701 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6bc42-e13a-43ea-b635-7756a1c79a9d admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:20.857975 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6bc42-e13a-43ea-b635-7756a1c79a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:20.858244 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6bc42-e13a-43ea-b635-7756a1c79a9d admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:20.859169 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f6bc42-e13a-43ea-b635-7756a1c79a9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:20.861381 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f6bc42-e13a-43ea-b635-7756a1c79a9d admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:20.862349 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccbc5ae-72a0-4ed4-a042-3225427555fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:20.864172 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccbc5ae-72a0-4ed4-a042-3225427555fb service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:20.872122 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1430/2849] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 23:56:20.874991 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1420/2850] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:56:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 23:56:20.876963 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f6bc42-e13a-43ea-b635-7756a1c79a9d admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:20.887317 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f6bc42-e13a-43ea-b635-7756a1c79a9d admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:20.892034 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccd90d4-ce1e-4a6a-b46e-28c719c6155f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:20.892675 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd90d4-ce1e-4a6a-b46e-28c719c6155f admin admin] REQUEST_METHOD: `PUT` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:20.892953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd90d4-ce1e-4a6a-b46e-28c719c6155f admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:20.893314 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd90d4-ce1e-4a6a-b46e-28c719c6155f admin admin] PATH_INFO: `/v3/projects/bdf54f0936e34221aff6469036ebf39c/users/de4ecf2000594b129b642ebd30bb4e6e/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:20.908403 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccd90d4-ce1e-4a6a-b46e-28c719c6155f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdf54f0936e34221aff6469036ebf39c, user_id=de4ecf2000594b129b642ebd30bb4e6e, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:20.911063 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccd90d4-ce1e-4a6a-b46e-28c719c6155f admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:20.924426 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1431/2851] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:56:20 2022] PUT /identity/v3/projects/bdf54f0936e34221aff6469036ebf39c/users/de4ecf2000594b129b642ebd30bb4e6e/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:56:20.937244 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c3a6bf-ebf9-46a9-9fe8-c491da5d33b0 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:20.937815 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c3a6bf-ebf9-46a9-9fe8-c491da5d33b0 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:20.938097 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c3a6bf-ebf9-46a9-9fe8-c491da5d33b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:20.967477 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-c8c3a6bf-ebf9-46a9-9fe8-c491da5d33b0 None None] MFA Rules not processed for user `de4ecf2000594b129b642ebd30bb4e6e`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:56:20.968782 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-c8c3a6bf-ebf9-46a9-9fe8-c491da5d33b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:21.035356 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1421/2852] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:56:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:56:21.046728 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccd90d4-ce1e-4a6a-b46e-28c719c6155f admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:21.048595 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-1ccd90d4-ce1e-4a6a-b46e-28c719c6155f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:21.084558 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:56:21.110092 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccd90d4-ce1e-4a6a-b46e-28c719c6155f admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:21.113759 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83592f25-051f-44b1-ba2c-1eda8ef228bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:21.114651 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83592f25-051f-44b1-ba2c-1eda8ef228bc service cinder] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:21.114651 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83592f25-051f-44b1-ba2c-1eda8ef228bc service cinder] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:21.114935 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83592f25-051f-44b1-ba2c-1eda8ef228bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:21.123857 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83592f25-051f-44b1-ba2c-1eda8ef228bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:21.125341 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83592f25-051f-44b1-ba2c-1eda8ef228bc service cinder] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:21.134803 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1432/2853] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:56:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:56:21.156378 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c3a6bf-ebf9-46a9-9fe8-c491da5d33b0 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:21.157894 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-c8c3a6bf-ebf9-46a9-9fe8-c491da5d33b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:21.215598 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=100 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:56:21.228888 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c3a6bf-ebf9-46a9-9fe8-c491da5d33b0 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:21.233416 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25996db2-fc0e-4a19-83d4-b8f35d687093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:21.234052 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25996db2-fc0e-4a19-83d4-b8f35d687093 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:21.234334 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25996db2-fc0e-4a19-83d4-b8f35d687093 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:21.234609 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25996db2-fc0e-4a19-83d4-b8f35d687093 admin admin] PATH_INFO: `/v3/projects/430c456402984ab2ad3e75e7213719f9` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:21.235745 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25996db2-fc0e-4a19-83d4-b8f35d687093 admin admin] RBAC: Authorizing `identity:delete_project(project_id=430c456402984ab2ad3e75e7213719f9)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:21.237689 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25996db2-fc0e-4a19-83d4-b8f35d687093 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:21.288273 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-25996db2-fc0e-4a19-83d4-b8f35d687093 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '430c456402984ab2ad3e75e7213719f9', 'request_id': 'req-25996db2-fc0e-4a19-83d4-b8f35d687093'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:21.294534 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-25996db2-fc0e-4a19-83d4-b8f35d687093 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '430c456402984ab2ad3e75e7213719f9', 'request_id': 'req-25996db2-fc0e-4a19-83d4-b8f35d687093'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:21.302178 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-25996db2-fc0e-4a19-83d4-b8f35d687093 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '430c456402984ab2ad3e75e7213719f9', 'request_id': 'req-25996db2-fc0e-4a19-83d4-b8f35d687093'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:21.314282 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-25996db2-fc0e-4a19-83d4-b8f35d687093 admin admin] The token cache is being invalidate because project 430c456402984ab2ad3e75e7213719f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:56:21.314591 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-25996db2-fc0e-4a19-83d4-b8f35d687093 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:21.315316 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-25996db2-fc0e-4a19-83d4-b8f35d687093 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:21.317090 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1422/2854] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:56:21 2022] DELETE /identity/v3/projects/430c456402984ab2ad3e75e7213719f9 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:56:21.321707 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83592f25-051f-44b1-ba2c-1eda8ef228bc service cinder] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:21.332830 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83592f25-051f-44b1-ba2c-1eda8ef228bc service cinder] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:21.393929 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc67929e-720f-48b5-8623-38a6bc07652f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:21.394576 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67929e-720f-48b5-8623-38a6bc07652f admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:21.394857 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67929e-720f-48b5-8623-38a6bc07652f admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:21.395132 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67929e-720f-48b5-8623-38a6bc07652f admin admin] PATH_INFO: `/v3/users/d3ca50c5b48b49ef9cddbb626d5a68ba` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:21.396342 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc67929e-720f-48b5-8623-38a6bc07652f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3ca50c5b48b49ef9cddbb626d5a68ba)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:21.398326 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc67929e-720f-48b5-8623-38a6bc07652f admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:21.457021 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-bc67929e-720f-48b5-8623-38a6bc07652f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3ca50c5b48b49ef9cddbb626d5a68ba'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:21.457698 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-bc67929e-720f-48b5-8623-38a6bc07652f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3ca50c5b48b49ef9cddbb626d5a68ba'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:21.472161 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-bc67929e-720f-48b5-8623-38a6bc07652f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3ca50c5b48b49ef9cddbb626d5a68ba'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:21.480495 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-bc67929e-720f-48b5-8623-38a6bc07652f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3ca50c5b48b49ef9cddbb626d5a68ba'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:21.481105 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-bc67929e-720f-48b5-8623-38a6bc07652f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3ca50c5b48b49ef9cddbb626d5a68ba'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:21.494748 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1433/2855] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:56:21 2022] DELETE /identity/v3/users/d3ca50c5b48b49ef9cddbb626d5a68ba => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:56:21.996247 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25996db2-fc0e-4a19-83d4-b8f35d687093 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:21.997699 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-25996db2-fc0e-4a19-83d4-b8f35d687093 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:22.012123 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25996db2-fc0e-4a19-83d4-b8f35d687093 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:22.069876 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2869f1-a557-496d-ba8b-6ff47ba98954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:22.070488 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2869f1-a557-496d-ba8b-6ff47ba98954 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:22.070766 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2869f1-a557-496d-ba8b-6ff47ba98954 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:22.071647 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2869f1-a557-496d-ba8b-6ff47ba98954 admin admin] PATH_INFO: `/v3/projects/67b7110eb2f64fb5a807205e20d8691e` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:22.072778 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2869f1-a557-496d-ba8b-6ff47ba98954 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67b7110eb2f64fb5a807205e20d8691e)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:22.074752 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2869f1-a557-496d-ba8b-6ff47ba98954 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:22.120373 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-fa2869f1-a557-496d-ba8b-6ff47ba98954 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67b7110eb2f64fb5a807205e20d8691e', 'request_id': 'req-fa2869f1-a557-496d-ba8b-6ff47ba98954'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:22.126204 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-fa2869f1-a557-496d-ba8b-6ff47ba98954 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67b7110eb2f64fb5a807205e20d8691e', 'request_id': 'req-fa2869f1-a557-496d-ba8b-6ff47ba98954'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:22.132949 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-fa2869f1-a557-496d-ba8b-6ff47ba98954 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67b7110eb2f64fb5a807205e20d8691e', 'request_id': 'req-fa2869f1-a557-496d-ba8b-6ff47ba98954'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:22.145494 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-fa2869f1-a557-496d-ba8b-6ff47ba98954 admin admin] The token cache is being invalidate because project 67b7110eb2f64fb5a807205e20d8691e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:56:22.145780 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-fa2869f1-a557-496d-ba8b-6ff47ba98954 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:22.146421 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-fa2869f1-a557-496d-ba8b-6ff47ba98954 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:22.147888 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1423/2856] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:56:21 2022] DELETE /identity/v3/projects/67b7110eb2f64fb5a807205e20d8691e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:56:22.152883 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ab9bf-99a8-4e91-9b9a-3e801cb354ff None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:22.153349 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ab9bf-99a8-4e91-9b9a-3e801cb354ff None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:22.153456 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ab9bf-99a8-4e91-9b9a-3e801cb354ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:22.172944 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-335ab9bf-99a8-4e91-9b9a-3e801cb354ff None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:56:22.174133 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-335ab9bf-99a8-4e91-9b9a-3e801cb354ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:22.220115 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:56:22.234427 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1434/2857] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:56:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:56:22.242275 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1c036-319d-4f64-9539-2bf2611cab6e None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:22.242506 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1c036-319d-4f64-9539-2bf2611cab6e None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:22.242784 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1c036-319d-4f64-9539-2bf2611cab6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:22.262380 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-3ac1c036-319d-4f64-9539-2bf2611cab6e None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:56:22.263583 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-3ac1c036-319d-4f64-9539-2bf2611cab6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:22.271558 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1424/2858] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:56:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:56:22.275721 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ab9bf-99a8-4e91-9b9a-3e801cb354ff None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:22.285448 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ab9bf-99a8-4e91-9b9a-3e801cb354ff None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:22.290546 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b778cb-3a30-4f23-baee-193a65cfec69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:22.291135 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b778cb-3a30-4f23-baee-193a65cfec69 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:22.291413 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b778cb-3a30-4f23-baee-193a65cfec69 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:22.291686 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b778cb-3a30-4f23-baee-193a65cfec69 admin admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:22.292359 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b778cb-3a30-4f23-baee-193a65cfec69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:56:22.292663 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b778cb-3a30-4f23-baee-193a65cfec69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:22.294579 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b778cb-3a30-4f23-baee-193a65cfec69 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:22.306476 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1435/2859] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 23:56:22.310802 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac1c036-319d-4f64-9539-2bf2611cab6e None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:22.320035 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac1c036-319d-4f64-9539-2bf2611cab6e None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:22.324700 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e762de8-d08a-4007-9bfa-fac32f4acf1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:22.325293 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e762de8-d08a-4007-9bfa-fac32f4acf1b admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:22.325571 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e762de8-d08a-4007-9bfa-fac32f4acf1b admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:22.325840 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e762de8-d08a-4007-9bfa-fac32f4acf1b admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:22.326455 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e762de8-d08a-4007-9bfa-fac32f4acf1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:22.328400 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e762de8-d08a-4007-9bfa-fac32f4acf1b admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:22.332788 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:56:22.357114 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1425/2860] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:56:22 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:56:22.361183 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b778cb-3a30-4f23-baee-193a65cfec69 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:22.371765 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b778cb-3a30-4f23-baee-193a65cfec69 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:22.426130 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b77e5f0-db9c-4101-ba6d-5f50c7cc586e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:22.426730 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77e5f0-db9c-4101-ba6d-5f50c7cc586e admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:22.427004 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77e5f0-db9c-4101-ba6d-5f50c7cc586e admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:22.427301 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77e5f0-db9c-4101-ba6d-5f50c7cc586e admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:22.427953 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b77e5f0-db9c-4101-ba6d-5f50c7cc586e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:22.429893 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b77e5f0-db9c-4101-ba6d-5f50c7cc586e admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:22.459221 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1436/2861] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:56:22 2022] POST /identity/v3/users => generated 284 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:56:22.460475 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e762de8-d08a-4007-9bfa-fac32f4acf1b admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:22.471166 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e762de8-d08a-4007-9bfa-fac32f4acf1b admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:22.475990 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d718c06d-26f7-407f-bb85-fbe138187cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:22.476594 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718c06d-26f7-407f-bb85-fbe138187cb2 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:22.476879 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718c06d-26f7-407f-bb85-fbe138187cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:22.477137 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718c06d-26f7-407f-bb85-fbe138187cb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:22.478292 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d718c06d-26f7-407f-bb85-fbe138187cb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:22.479871 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d718c06d-26f7-407f-bb85-fbe138187cb2 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:22.492775 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1426/2862] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 23:56:22.494970 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b77e5f0-db9c-4101-ba6d-5f50c7cc586e admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:22.505470 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b77e5f0-db9c-4101-ba6d-5f50c7cc586e admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:22.510400 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12872cbe-6c7d-4d52-aac8-4e30083c4689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:22.510992 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12872cbe-6c7d-4d52-aac8-4e30083c4689 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:22.511268 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12872cbe-6c7d-4d52-aac8-4e30083c4689 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:22.511586 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12872cbe-6c7d-4d52-aac8-4e30083c4689 admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:22.512273 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12872cbe-6c7d-4d52-aac8-4e30083c4689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:22.513924 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12872cbe-6c7d-4d52-aac8-4e30083c4689 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:22.525022 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1437/2863] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:56:22.529221 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d718c06d-26f7-407f-bb85-fbe138187cb2 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:22.541014 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d718c06d-26f7-407f-bb85-fbe138187cb2 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:22.545990 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf49b3-b4f7-44ac-9885-c1aa0cf33c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:22.546625 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf49b3-b4f7-44ac-9885-c1aa0cf33c1b admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:22.546899 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf49b3-b4f7-44ac-9885-c1aa0cf33c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:22.547167 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf49b3-b4f7-44ac-9885-c1aa0cf33c1b admin admin] PATH_INFO: `/v3/projects/aec59b0322094f8098ab0690fced46d5/users/d9c11ba067614830a9861dbe5d79a722/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:22.562852 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf49b3-b4f7-44ac-9885-c1aa0cf33c1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=aec59b0322094f8098ab0690fced46d5, user_id=d9c11ba067614830a9861dbe5d79a722, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:22.565541 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf49b3-b4f7-44ac-9885-c1aa0cf33c1b admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:22.579453 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1427/2864] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:56:22 2022] PUT /identity/v3/projects/aec59b0322094f8098ab0690fced46d5/users/d9c11ba067614830a9861dbe5d79a722/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:56:22.590890 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019fa0b-e61f-4b0f-b082-c04ceee20d35 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:22.591177 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019fa0b-e61f-4b0f-b082-c04ceee20d35 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:22.591446 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019fa0b-e61f-4b0f-b082-c04ceee20d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:22.620386 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-9019fa0b-e61f-4b0f-b082-c04ceee20d35 None None] MFA Rules not processed for user `d9c11ba067614830a9861dbe5d79a722`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:56:22.621580 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-9019fa0b-e61f-4b0f-b082-c04ceee20d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:22.694164 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1438/2865] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:56:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:56:22.705496 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf49b3-b4f7-44ac-9885-c1aa0cf33c1b admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:22.707196 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-e8cf49b3-b4f7-44ac-9885-c1aa0cf33c1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:22.769316 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf49b3-b4f7-44ac-9885-c1aa0cf33c1b admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:22.773312 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18606432-3bca-43bc-9920-b49337f98ee6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62906d50b6e247c59b69809d3959d4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:22.773921 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18606432-3bca-43bc-9920-b49337f98ee6 service cinder] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:22.774760 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18606432-3bca-43bc-9920-b49337f98ee6 service cinder] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:22.775034 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18606432-3bca-43bc-9920-b49337f98ee6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:22.783692 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18606432-3bca-43bc-9920-b49337f98ee6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:22.785161 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18606432-3bca-43bc-9920-b49337f98ee6 service cinder] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:22.794758 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1428/2866] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:56:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:56:22.819473 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9019fa0b-e61f-4b0f-b082-c04ceee20d35 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:22.870476 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:56:22.886724 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9019fa0b-e61f-4b0f-b082-c04ceee20d35 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:22.891329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef2b92-6b40-45ee-bcc3-9b6dce43cdcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:22.891952 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef2b92-6b40-45ee-bcc3-9b6dce43cdcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:22.892257 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef2b92-6b40-45ee-bcc3-9b6dce43cdcf admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:22.892525 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef2b92-6b40-45ee-bcc3-9b6dce43cdcf admin admin] PATH_INFO: `/v3/users/d9c11ba067614830a9861dbe5d79a722` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:22.893658 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ef2b92-6b40-45ee-bcc3-9b6dce43cdcf admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9c11ba067614830a9861dbe5d79a722)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:22.895553 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ef2b92-6b40-45ee-bcc3-9b6dce43cdcf admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:22.943676 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-17ef2b92-6b40-45ee-bcc3-9b6dce43cdcf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9c11ba067614830a9861dbe5d79a722'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:22.944324 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-17ef2b92-6b40-45ee-bcc3-9b6dce43cdcf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9c11ba067614830a9861dbe5d79a722'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:22.956538 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-17ef2b92-6b40-45ee-bcc3-9b6dce43cdcf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9c11ba067614830a9861dbe5d79a722'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:22.964957 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-17ef2b92-6b40-45ee-bcc3-9b6dce43cdcf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9c11ba067614830a9861dbe5d79a722'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:22.965745 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-17ef2b92-6b40-45ee-bcc3-9b6dce43cdcf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9c11ba067614830a9861dbe5d79a722'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:22.979010 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1439/2867] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:56:22 2022] DELETE /identity/v3/users/d9c11ba067614830a9861dbe5d79a722 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:56:22.983501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d0b37-39d4-4c82-8f4a-7da568e2b415 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:22.983789 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d0b37-39d4-4c82-8f4a-7da568e2b415 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:22.984093 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d0b37-39d4-4c82-8f4a-7da568e2b415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:23.002647 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-0c2d0b37-39d4-4c82-8f4a-7da568e2b415 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:56:23.003903 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-0c2d0b37-39d4-4c82-8f4a-7da568e2b415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:23.064859 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1429/2868] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:56:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:56:23.075368 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef2b92-6b40-45ee-bcc3-9b6dce43cdcf admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:23.076945 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-17ef2b92-6b40-45ee-bcc3-9b6dce43cdcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:23.137716 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef2b92-6b40-45ee-bcc3-9b6dce43cdcf admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:23.141534 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59f91e-5e82-4f06-9e30-e9452737f50f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:23.142304 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59f91e-5e82-4f06-9e30-e9452737f50f service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:23.142593 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59f91e-5e82-4f06-9e30-e9452737f50f service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:23.142915 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59f91e-5e82-4f06-9e30-e9452737f50f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:23.152910 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59f91e-5e82-4f06-9e30-e9452737f50f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:23.154353 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59f91e-5e82-4f06-9e30-e9452737f50f service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:23.164188 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1440/2869] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:56:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 23:56:23.427218 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d0b37-39d4-4c82-8f4a-7da568e2b415 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:23.428927 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-0c2d0b37-39d4-4c82-8f4a-7da568e2b415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:23.443664 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d0b37-39d4-4c82-8f4a-7da568e2b415 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:23.449786 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e13c5a-f972-4e3f-9328-fc700c17183b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:23.450422 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e13c5a-f972-4e3f-9328-fc700c17183b admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:23.450695 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e13c5a-f972-4e3f-9328-fc700c17183b admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:23.450966 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e13c5a-f972-4e3f-9328-fc700c17183b admin admin] PATH_INFO: `/v3/projects/aec59b0322094f8098ab0690fced46d5` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:23.452830 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e13c5a-f972-4e3f-9328-fc700c17183b admin admin] RBAC: Authorizing `identity:delete_project(project_id=aec59b0322094f8098ab0690fced46d5)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:23.455289 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e13c5a-f972-4e3f-9328-fc700c17183b admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:23.501497 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:56:23.502741 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-62e13c5a-f972-4e3f-9328-fc700c17183b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aec59b0322094f8098ab0690fced46d5', 'request_id': 'req-62e13c5a-f972-4e3f-9328-fc700c17183b'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:23.510166 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-62e13c5a-f972-4e3f-9328-fc700c17183b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aec59b0322094f8098ab0690fced46d5', 'request_id': 'req-62e13c5a-f972-4e3f-9328-fc700c17183b'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:23.517351 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-62e13c5a-f972-4e3f-9328-fc700c17183b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aec59b0322094f8098ab0690fced46d5', 'request_id': 'req-62e13c5a-f972-4e3f-9328-fc700c17183b'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:23.528514 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-62e13c5a-f972-4e3f-9328-fc700c17183b admin admin] The token cache is being invalidate because project aec59b0322094f8098ab0690fced46d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:56:23.528834 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-62e13c5a-f972-4e3f-9328-fc700c17183b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:23.529468 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-62e13c5a-f972-4e3f-9328-fc700c17183b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:23.530942 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1430/2870] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:56:23 2022] DELETE /identity/v3/projects/aec59b0322094f8098ab0690fced46d5 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:56:24.172254 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59f91e-5e82-4f06-9e30-e9452737f50f service neutron] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:24.173634 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-0c59f91e-5e82-4f06-9e30-e9452737f50f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:24.186101 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59f91e-5e82-4f06-9e30-e9452737f50f service neutron] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:24.240846 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e9cb7-f3ec-47b3-8ec2-a1a8d93e9f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:24.241460 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e9cb7-f3ec-47b3-8ec2-a1a8d93e9f6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:24.241736 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e9cb7-f3ec-47b3-8ec2-a1a8d93e9f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:24.242006 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e9cb7-f3ec-47b3-8ec2-a1a8d93e9f6e admin admin] PATH_INFO: `/v3/users/de4ecf2000594b129b642ebd30bb4e6e` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:24.243148 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31e9cb7-f3ec-47b3-8ec2-a1a8d93e9f6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=de4ecf2000594b129b642ebd30bb4e6e)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:24.245090 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31e9cb7-f3ec-47b3-8ec2-a1a8d93e9f6e admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:24.256842 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:56:24.290450 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-d31e9cb7-f3ec-47b3-8ec2-a1a8d93e9f6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de4ecf2000594b129b642ebd30bb4e6e'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:24.291121 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-d31e9cb7-f3ec-47b3-8ec2-a1a8d93e9f6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de4ecf2000594b129b642ebd30bb4e6e'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:24.302942 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-d31e9cb7-f3ec-47b3-8ec2-a1a8d93e9f6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de4ecf2000594b129b642ebd30bb4e6e'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:24.309607 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-d31e9cb7-f3ec-47b3-8ec2-a1a8d93e9f6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de4ecf2000594b129b642ebd30bb4e6e'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:24.310206 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-d31e9cb7-f3ec-47b3-8ec2-a1a8d93e9f6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de4ecf2000594b129b642ebd30bb4e6e'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:24.322014 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1441/2871] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:56:24 2022] DELETE /identity/v3/users/de4ecf2000594b129b642ebd30bb4e6e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:56:24.326914 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2780dc-9619-4fcc-bf67-f03061af3899 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:24.327403 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2780dc-9619-4fcc-bf67-f03061af3899 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:24.327527 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2780dc-9619-4fcc-bf67-f03061af3899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:24.346961 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-be2780dc-9619-4fcc-bf67-f03061af3899 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:56:24.348196 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-be2780dc-9619-4fcc-bf67-f03061af3899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:24.407066 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1431/2872] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:56:24.417237 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e9cb7-f3ec-47b3-8ec2-a1a8d93e9f6e admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:24.418895 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-d31e9cb7-f3ec-47b3-8ec2-a1a8d93e9f6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:24.476094 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e9cb7-f3ec-47b3-8ec2-a1a8d93e9f6e admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:24.480304 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1fb2c7-65cb-482f-8fec-b2707b898c4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:24.480909 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1fb2c7-65cb-482f-8fec-b2707b898c4f service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:24.481183 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1fb2c7-65cb-482f-8fec-b2707b898c4f service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:24.481449 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1fb2c7-65cb-482f-8fec-b2707b898c4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:24.490291 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1fb2c7-65cb-482f-8fec-b2707b898c4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:24.491734 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1fb2c7-65cb-482f-8fec-b2707b898c4f service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:24.501584 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1442/2873] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:56:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:56:24.788879 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2780dc-9619-4fcc-bf67-f03061af3899 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:24.790339 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-be2780dc-9619-4fcc-bf67-f03061af3899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:24.802443 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2780dc-9619-4fcc-bf67-f03061af3899 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:24.807030 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce58204c-e80f-4e2e-a4d7-a05aeee95393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:24.807653 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58204c-e80f-4e2e-a4d7-a05aeee95393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:24.807920 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58204c-e80f-4e2e-a4d7-a05aeee95393 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:24.808235 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58204c-e80f-4e2e-a4d7-a05aeee95393 admin admin] PATH_INFO: `/v3/projects/bdf54f0936e34221aff6469036ebf39c` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:24.809346 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce58204c-e80f-4e2e-a4d7-a05aeee95393 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdf54f0936e34221aff6469036ebf39c)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:24.811253 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce58204c-e80f-4e2e-a4d7-a05aeee95393 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:24.857963 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-ce58204c-e80f-4e2e-a4d7-a05aeee95393 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdf54f0936e34221aff6469036ebf39c', 'request_id': 'req-ce58204c-e80f-4e2e-a4d7-a05aeee95393'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:24.863585 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-ce58204c-e80f-4e2e-a4d7-a05aeee95393 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdf54f0936e34221aff6469036ebf39c', 'request_id': 'req-ce58204c-e80f-4e2e-a4d7-a05aeee95393'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:24.870954 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-ce58204c-e80f-4e2e-a4d7-a05aeee95393 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdf54f0936e34221aff6469036ebf39c', 'request_id': 'req-ce58204c-e80f-4e2e-a4d7-a05aeee95393'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:24.883143 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-ce58204c-e80f-4e2e-a4d7-a05aeee95393 admin admin] The token cache is being invalidate because project bdf54f0936e34221aff6469036ebf39c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85834) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:56:24.883425 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-ce58204c-e80f-4e2e-a4d7-a05aeee95393 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:24.884602 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-ce58204c-e80f-4e2e-a4d7-a05aeee95393 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:56:24.886060 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1432/2874] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:56:24 2022] DELETE /identity/v3/projects/bdf54f0936e34221aff6469036ebf39c => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:56:24.891756 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05351bce-1e41-461e-8d6d-1104e369884b None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:24.892059 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05351bce-1e41-461e-8d6d-1104e369884b None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:24.892329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05351bce-1e41-461e-8d6d-1104e369884b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:24.911385 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-05351bce-1e41-461e-8d6d-1104e369884b None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:56:24.912528 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-05351bce-1e41-461e-8d6d-1104e369884b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:24.969511 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1443/2875] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:56:24.977300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754dda4e-89ad-4271-8a6b-5eff6653fd86 None None] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:24.977588 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754dda4e-89ad-4271-8a6b-5eff6653fd86 None None] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:24.977866 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754dda4e-89ad-4271-8a6b-5eff6653fd86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:24.997685 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.auth.core [None req-754dda4e-89ad-4271-8a6b-5eff6653fd86 None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85834) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:56:24.998895 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-754dda4e-89ad-4271-8a6b-5eff6653fd86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:25.006749 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1433/2876] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:56:25.011499 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05351bce-1e41-461e-8d6d-1104e369884b None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:25.020978 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05351bce-1e41-461e-8d6d-1104e369884b None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:25.025448 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93025ed1-5809-41b0-9810-f192ac4a62e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:25.026027 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93025ed1-5809-41b0-9810-f192ac4a62e7 admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:25.026300 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93025ed1-5809-41b0-9810-f192ac4a62e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:25.026564 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93025ed1-5809-41b0-9810-f192ac4a62e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:25.027198 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93025ed1-5809-41b0-9810-f192ac4a62e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=85835) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 23:56:25.027497 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93025ed1-5809-41b0-9810-f192ac4a62e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:25.029123 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93025ed1-5809-41b0-9810-f192ac4a62e7 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:25.040203 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1444/2877] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Aug 17 23:56:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 23:56:25.044465 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754dda4e-89ad-4271-8a6b-5eff6653fd86 None None] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:25.053709 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754dda4e-89ad-4271-8a6b-5eff6653fd86 None None] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:25.058823 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca34cb72-c22f-4a3c-b228-d18bd3fe1d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:25.059415 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34cb72-c22f-4a3c-b228-d18bd3fe1d4f admin admin] REQUEST_METHOD: `POST` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:25.059699 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34cb72-c22f-4a3c-b228-d18bd3fe1d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:25.059995 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34cb72-c22f-4a3c-b228-d18bd3fe1d4f admin admin] PATH_INFO: `/v3/projects` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:25.060633 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca34cb72-c22f-4a3c-b228-d18bd3fe1d4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:25.062589 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca34cb72-c22f-4a3c-b228-d18bd3fe1d4f admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:25.089918 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1434/2878] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Aug 17 23:56:25 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 23:56:25.094089 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93025ed1-5809-41b0-9810-f192ac4a62e7 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:25.103888 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93025ed1-5809-41b0-9810-f192ac4a62e7 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:25.120601 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:56:25.160840 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb092be3-8e88-4469-9082-c0d0e9656a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:25.161492 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb092be3-8e88-4469-9082-c0d0e9656a0b admin admin] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:25.161723 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb092be3-8e88-4469-9082-c0d0e9656a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:25.161998 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb092be3-8e88-4469-9082-c0d0e9656a0b admin admin] PATH_INFO: `/v3/users` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:25.162613 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb092be3-8e88-4469-9082-c0d0e9656a0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:25.164576 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb092be3-8e88-4469-9082-c0d0e9656a0b admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:25.185460 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1445/2879] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Aug 17 23:56:25 2022] POST /identity/v3/users => generated 284 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 23:56:25.189587 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca34cb72-c22f-4a3c-b228-d18bd3fe1d4f admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:25.200295 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca34cb72-c22f-4a3c-b228-d18bd3fe1d4f admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:25.204877 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3611b12e-eb78-4b10-9cf7-62f731ddcd97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:25.205468 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611b12e-eb78-4b10-9cf7-62f731ddcd97 admin admin] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:25.205747 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611b12e-eb78-4b10-9cf7-62f731ddcd97 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:25.206021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611b12e-eb78-4b10-9cf7-62f731ddcd97 admin admin] PATH_INFO: `/v3/roles` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:25.206662 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3611b12e-eb78-4b10-9cf7-62f731ddcd97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:25.208273 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3611b12e-eb78-4b10-9cf7-62f731ddcd97 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:25.222838 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1435/2880] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:56:25.226923 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb092be3-8e88-4469-9082-c0d0e9656a0b admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:25.237246 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb092be3-8e88-4469-9082-c0d0e9656a0b admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:25.241751 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d730c834-e362-4c13-89a2-c2aefd3e7aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:25.242634 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730c834-e362-4c13-89a2-c2aefd3e7aec admin admin] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:25.242942 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730c834-e362-4c13-89a2-c2aefd3e7aec admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:25.243707 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730c834-e362-4c13-89a2-c2aefd3e7aec admin admin] PATH_INFO: `/v3/roles` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:25.244368 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d730c834-e362-4c13-89a2-c2aefd3e7aec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:25.245946 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d730c834-e362-4c13-89a2-c2aefd3e7aec admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:25.256238 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1446/2881] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Aug 17 23:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 23:56:25.260085 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3611b12e-eb78-4b10-9cf7-62f731ddcd97 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:25.270059 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3611b12e-eb78-4b10-9cf7-62f731ddcd97 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:25.274677 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af0a7f-2fb3-417c-80fe-2817ab7f0172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:25.275341 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af0a7f-2fb3-417c-80fe-2817ab7f0172 admin admin] REQUEST_METHOD: `PUT` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:25.275618 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af0a7f-2fb3-417c-80fe-2817ab7f0172 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:25.275886 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af0a7f-2fb3-417c-80fe-2817ab7f0172 admin admin] PATH_INFO: `/v3/projects/bc3dd0bb0a3f467cb7c038bb6bdaaa3b/users/5366cc8ce17343328d602023ffe22846/roles/1182317a465349478dde4e28cb7cf96f` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:25.290738 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28af0a7f-2fb3-417c-80fe-2817ab7f0172 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc3dd0bb0a3f467cb7c038bb6bdaaa3b, user_id=5366cc8ce17343328d602023ffe22846, role_id=1182317a465349478dde4e28cb7cf96f)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:25.293748 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28af0a7f-2fb3-417c-80fe-2817ab7f0172 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:25.296704 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:56:25.310097 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1436/2882] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Aug 17 23:56:25 2022] PUT /identity/v3/projects/bc3dd0bb0a3f467cb7c038bb6bdaaa3b/users/5366cc8ce17343328d602023ffe22846/roles/1182317a465349478dde4e28cb7cf96f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:56:25.320530 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114821fd-3b8c-4c6c-845d-6427da042e39 None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:25.320819 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114821fd-3b8c-4c6c-845d-6427da042e39 None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:25.321089 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114821fd-3b8c-4c6c-845d-6427da042e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:25.349128 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-114821fd-3b8c-4c6c-845d-6427da042e39 None None] MFA Rules not processed for user `5366cc8ce17343328d602023ffe22846`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:56:25.350329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-114821fd-3b8c-4c6c-845d-6427da042e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:25.412430 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1447/2883] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:56:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 23:56:25.425274 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af0a7f-2fb3-417c-80fe-2817ab7f0172 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:25.427077 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-28af0a7f-2fb3-417c-80fe-2817ab7f0172 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:25.505761 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af0a7f-2fb3-417c-80fe-2817ab7f0172 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:25.509424 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab4cbe-ce21-467f-83ab-c5f91c3c836b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c08c497a14c426f9cfe0dd8c8100a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:25.510021 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab4cbe-ce21-467f-83ab-c5f91c3c836b service glance] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:25.510300 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab4cbe-ce21-467f-83ab-c5f91c3c836b service glance] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:25.510570 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab4cbe-ce21-467f-83ab-c5f91c3c836b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:25.518813 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab4cbe-ce21-467f-83ab-c5f91c3c836b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:25.520268 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab4cbe-ce21-467f-83ab-c5f91c3c836b service glance] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:25.528921 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1437/2884] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:56:25 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 23:56:35.409836 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=57,keystone:INSERT=3 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:56:35.525165 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=37 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:56:52.660354 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114821fd-3b8c-4c6c-845d-6427da042e39 None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:56:52.662820 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-114821fd-3b8c-4c6c-845d-6427da042e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:52.746245 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114821fd-3b8c-4c6c-845d-6427da042e39 None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:56:52.752477 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd22fdf5-5a8d-4b9d-9aa4-6e69ade5dfb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec1dcbe43b0749c3bcc4af9100d771d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:56:52.753087 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22fdf5-5a8d-4b9d-9aa4-6e69ade5dfb2 service nova] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:56:52.753363 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22fdf5-5a8d-4b9d-9aa4-6e69ade5dfb2 service nova] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:56:52.753631 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22fdf5-5a8d-4b9d-9aa4-6e69ade5dfb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:56:52.755535 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-fd22fdf5-5a8d-4b9d-9aa4-6e69ade5dfb2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:56:52.769564 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd22fdf5-5a8d-4b9d-9aa4-6e69ade5dfb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:56:52.772196 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd22fdf5-5a8d-4b9d-9aa4-6e69ade5dfb2 service nova] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:56:52.791829 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1448/2885] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:56:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:57:02.784660 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=34 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:57:12.641681 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab4cbe-ce21-467f-83ab-c5f91c3c836b service glance] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:57:12.643321 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-7eab4cbe-ce21-467f-83ab-c5f91c3c836b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:57:12.715183 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab4cbe-ce21-467f-83ab-c5f91c3c836b service glance] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:57:12.720747 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae4be7-fb71-4bb2-98ea-00a7c92b2856 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc57d911a56423e96c736bc96270ba8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:57:12.721347 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae4be7-fb71-4bb2-98ea-00a7c92b2856 service placement] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:57:12.721628 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae4be7-fb71-4bb2-98ea-00a7c92b2856 service placement] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:57:12.721902 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae4be7-fb71-4bb2-98ea-00a7c92b2856 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:57:12.723614 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-aeae4be7-fb71-4bb2-98ea-00a7c92b2856 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:57:12.735221 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae4be7-fb71-4bb2-98ea-00a7c92b2856 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:57:12.736933 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae4be7-fb71-4bb2-98ea-00a7c92b2856 service placement] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:57:12.747063 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1438/2886] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:57:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 23:57:16.195885 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd22fdf5-5a8d-4b9d-9aa4-6e69ade5dfb2 service nova] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:57:16.197332 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-fd22fdf5-5a8d-4b9d-9aa4-6e69ade5dfb2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:57:16.253188 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd22fdf5-5a8d-4b9d-9aa4-6e69ade5dfb2 service nova] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:57:16.256799 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f32c0e-5fe4-4d8a-a3b6-9a83195102f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:57:16.257391 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f32c0e-5fe4-4d8a-a3b6-9a83195102f4 service neutron] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:57:16.257671 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f32c0e-5fe4-4d8a-a3b6-9a83195102f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:57:16.257945 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f32c0e-5fe4-4d8a-a3b6-9a83195102f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:57:16.259651 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-61f32c0e-5fe4-4d8a-a3b6-9a83195102f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:57:16.321888 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f32c0e-5fe4-4d8a-a3b6-9a83195102f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:57:16.323258 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f32c0e-5fe4-4d8a-a3b6-9a83195102f4 service neutron] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:57:16.333402 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1449/2887] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:57:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:57:16.871746 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae4be7-fb71-4bb2-98ea-00a7c92b2856 service placement] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:57:16.882177 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae4be7-fb71-4bb2-98ea-00a7c92b2856 service placement] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:57:16.886640 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718d753-db45-44e1-8dc0-ecba21d1dae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:57:16.887306 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718d753-db45-44e1-8dc0-ecba21d1dae9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:57:16.887580 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718d753-db45-44e1-8dc0-ecba21d1dae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:57:16.887851 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718d753-db45-44e1-8dc0-ecba21d1dae9 admin admin] PATH_INFO: `/v3/users/5366cc8ce17343328d602023ffe22846` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:57:16.888989 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5718d753-db45-44e1-8dc0-ecba21d1dae9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5366cc8ce17343328d602023ffe22846)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:57:16.890946 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5718d753-db45-44e1-8dc0-ecba21d1dae9 admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:57:16.938154 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-5718d753-db45-44e1-8dc0-ecba21d1dae9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5366cc8ce17343328d602023ffe22846'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:57:16.938817 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-5718d753-db45-44e1-8dc0-ecba21d1dae9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5366cc8ce17343328d602023ffe22846'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:57:16.939428 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-5718d753-db45-44e1-8dc0-ecba21d1dae9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5366cc8ce17343328d602023ffe22846'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:57:16.946946 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-5718d753-db45-44e1-8dc0-ecba21d1dae9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5366cc8ce17343328d602023ffe22846'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:57:16.954639 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-5718d753-db45-44e1-8dc0-ecba21d1dae9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5366cc8ce17343328d602023ffe22846'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:57:16.973183 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1439/2888] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:57:16 2022] DELETE /identity/v3/users/5366cc8ce17343328d602023ffe22846 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:57:16.977700 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040529da-d7de-49c1-b7c7-79d6b26c2bab None None] REQUEST_METHOD: `POST` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:57:16.978085 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040529da-d7de-49c1-b7c7-79d6b26c2bab None None] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:57:16.978385 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040529da-d7de-49c1-b7c7-79d6b26c2bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:57:16.997564 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.auth.core [None req-040529da-d7de-49c1-b7c7-79d6b26c2bab None None] MFA Rules not processed for user `8f1a8fbc56c44c829c8dd8013a782220`. Rule list: `[]` (Enabled: `True`). {{(pid=85835) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 23:57:16.998766 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-040529da-d7de-49c1-b7c7-79d6b26c2bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:57:17.057078 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1450/2889] 10.222.0.26 () {58 vars in 958 bytes} [Wed Aug 17 23:57:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 23:57:17.067414 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718d753-db45-44e1-8dc0-ecba21d1dae9 admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:57:17.068879 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-5718d753-db45-44e1-8dc0-ecba21d1dae9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:57:17.138965 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718d753-db45-44e1-8dc0-ecba21d1dae9 admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:57:17.143781 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092889da-ae61-411a-beb4-7837a657a296 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16e2758f1cd144a98b98d9be0fa46ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:57:17.144549 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092889da-ae61-411a-beb4-7837a657a296 service neutron] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:57:17.144835 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092889da-ae61-411a-beb4-7837a657a296 service neutron] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:57:17.145110 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092889da-ae61-411a-beb4-7837a657a296 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:57:17.154726 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092889da-ae61-411a-beb4-7837a657a296 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:57:17.156344 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092889da-ae61-411a-beb4-7837a657a296 service neutron] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:57:17.168711 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1440/2890] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:57:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:57:17.510591 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040529da-d7de-49c1-b7c7-79d6b26c2bab None None] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:57:17.512256 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-040529da-d7de-49c1-b7c7-79d6b26c2bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:57:17.533373 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040529da-d7de-49c1-b7c7-79d6b26c2bab None None] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:57:17.538385 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b830ad-d4e3-4bfb-acd1-8ee355b59c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:57:17.539041 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b830ad-d4e3-4bfb-acd1-8ee355b59c57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:57:17.539347 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b830ad-d4e3-4bfb-acd1-8ee355b59c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:57:17.539581 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b830ad-d4e3-4bfb-acd1-8ee355b59c57 admin admin] PATH_INFO: `/v3/projects/bc3dd0bb0a3f467cb7c038bb6bdaaa3b` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:57:17.541672 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b830ad-d4e3-4bfb-acd1-8ee355b59c57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc3dd0bb0a3f467cb7c038bb6bdaaa3b)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:57:17.545291 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b830ad-d4e3-4bfb-acd1-8ee355b59c57 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:57:17.594888 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:57:17.597543 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-21b830ad-d4e3-4bfb-acd1-8ee355b59c57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc3dd0bb0a3f467cb7c038bb6bdaaa3b', 'request_id': 'req-21b830ad-d4e3-4bfb-acd1-8ee355b59c57'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:57:17.604208 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-21b830ad-d4e3-4bfb-acd1-8ee355b59c57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc3dd0bb0a3f467cb7c038bb6bdaaa3b', 'request_id': 'req-21b830ad-d4e3-4bfb-acd1-8ee355b59c57'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:57:17.612219 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-21b830ad-d4e3-4bfb-acd1-8ee355b59c57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc3dd0bb0a3f467cb7c038bb6bdaaa3b', 'request_id': 'req-21b830ad-d4e3-4bfb-acd1-8ee355b59c57'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:57:17.625114 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-21b830ad-d4e3-4bfb-acd1-8ee355b59c57 admin admin] The token cache is being invalidate because project bc3dd0bb0a3f467cb7c038bb6bdaaa3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:57:17.626110 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-21b830ad-d4e3-4bfb-acd1-8ee355b59c57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:57:17.626831 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-21b830ad-d4e3-4bfb-acd1-8ee355b59c57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:57:17.628379 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1451/2891] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:57:17 2022] DELETE /identity/v3/projects/bc3dd0bb0a3f467cb7c038bb6bdaaa3b => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:57:19.460057 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092889da-ae61-411a-beb4-7837a657a296 service neutron] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:57:19.461435 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-092889da-ae61-411a-beb4-7837a657a296 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:57:19.467926 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:57:19.546644 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092889da-ae61-411a-beb4-7837a657a296 service neutron] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:57:19.614853 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad676346-f53c-43c3-b04b-cabe6109adff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc57d911a56423e96c736bc96270ba8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:57:19.615718 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad676346-f53c-43c3-b04b-cabe6109adff service placement] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:57:19.616169 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad676346-f53c-43c3-b04b-cabe6109adff service placement] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:57:19.616540 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad676346-f53c-43c3-b04b-cabe6109adff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:57:19.618501 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-ad676346-f53c-43c3-b04b-cabe6109adff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:57:19.634640 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad676346-f53c-43c3-b04b-cabe6109adff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:57:19.636260 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad676346-f53c-43c3-b04b-cabe6109adff service placement] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:57:19.646822 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1441/2892] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:57:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:57:22.576680 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b830ad-d4e3-4bfb-acd1-8ee355b59c57 admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:57:22.578494 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-21b830ad-d4e3-4bfb-acd1-8ee355b59c57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:57:22.599766 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b830ad-d4e3-4bfb-acd1-8ee355b59c57 admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:57:22.605262 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca6f398-35b1-4477-a3f4-97a75bf6ffe6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc57d911a56423e96c736bc96270ba8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:57:22.605947 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca6f398-35b1-4477-a3f4-97a75bf6ffe6 service placement] REQUEST_METHOD: `GET` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:57:22.606238 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca6f398-35b1-4477-a3f4-97a75bf6ffe6 service placement] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:57:22.606511 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca6f398-35b1-4477-a3f4-97a75bf6ffe6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:57:22.608529 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-cca6f398-35b1-4477-a3f4-97a75bf6ffe6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:57:22.624726 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca6f398-35b1-4477-a3f4-97a75bf6ffe6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:57:22.627266 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca6f398-35b1-4477-a3f4-97a75bf6ffe6 service placement] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:57:22.639617 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1452/2893] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Aug 17 23:57:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 23:57:22.644649 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad676346-f53c-43c3-b04b-cabe6109adff service placement] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:57:22.646234 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-ad676346-f53c-43c3-b04b-cabe6109adff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:57:22.721499 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad676346-f53c-43c3-b04b-cabe6109adff service placement] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:57:22.726512 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a888a83-7a72-4122-9ba9-ab592d9cbaef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:57:22.727142 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a888a83-7a72-4122-9ba9-ab592d9cbaef admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:57:22.727424 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a888a83-7a72-4122-9ba9-ab592d9cbaef admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:57:22.727692 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a888a83-7a72-4122-9ba9-ab592d9cbaef admin admin] PATH_INFO: `/v3/users/cf6e346e96144628ac25c428be38870a` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:57:22.742622 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a888a83-7a72-4122-9ba9-ab592d9cbaef admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf6e346e96144628ac25c428be38870a)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:57:22.744578 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a888a83-7a72-4122-9ba9-ab592d9cbaef admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:57:22.772174 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:57:22.790186 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-9a888a83-7a72-4122-9ba9-ab592d9cbaef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf6e346e96144628ac25c428be38870a'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:57:22.790837 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-9a888a83-7a72-4122-9ba9-ab592d9cbaef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf6e346e96144628ac25c428be38870a'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:57:22.791451 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-9a888a83-7a72-4122-9ba9-ab592d9cbaef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf6e346e96144628ac25c428be38870a'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:57:22.799244 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-9a888a83-7a72-4122-9ba9-ab592d9cbaef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf6e346e96144628ac25c428be38870a'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:57:22.806307 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-9a888a83-7a72-4122-9ba9-ab592d9cbaef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf6e346e96144628ac25c428be38870a'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:57:22.823894 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1442/2894] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:57:22 2022] DELETE /identity/v3/users/cf6e346e96144628ac25c428be38870a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 23:57:22.996826 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca6f398-35b1-4477-a3f4-97a75bf6ffe6 service placement] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:57:23.011476 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca6f398-35b1-4477-a3f4-97a75bf6ffe6 service placement] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:57:23.073103 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c969972e-e11e-4b20-9eee-b4e10a58d48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:57:23.073745 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969972e-e11e-4b20-9eee-b4e10a58d48b admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:57:23.074022 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969972e-e11e-4b20-9eee-b4e10a58d48b admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:57:23.074290 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969972e-e11e-4b20-9eee-b4e10a58d48b admin admin] PATH_INFO: `/v3/projects/8e81ed7b06f84b2bab3369334cb1f90a` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:57:23.086261 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c969972e-e11e-4b20-9eee-b4e10a58d48b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e81ed7b06f84b2bab3369334cb1f90a)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:57:23.088232 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c969972e-e11e-4b20-9eee-b4e10a58d48b admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:57:23.153269 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-c969972e-e11e-4b20-9eee-b4e10a58d48b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e81ed7b06f84b2bab3369334cb1f90a', 'request_id': 'req-c969972e-e11e-4b20-9eee-b4e10a58d48b'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:57:23.159824 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-c969972e-e11e-4b20-9eee-b4e10a58d48b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e81ed7b06f84b2bab3369334cb1f90a', 'request_id': 'req-c969972e-e11e-4b20-9eee-b4e10a58d48b'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:57:23.167643 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-c969972e-e11e-4b20-9eee-b4e10a58d48b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e81ed7b06f84b2bab3369334cb1f90a', 'request_id': 'req-c969972e-e11e-4b20-9eee-b4e10a58d48b'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:57:23.180514 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-c969972e-e11e-4b20-9eee-b4e10a58d48b admin admin] The token cache is being invalidate because project 8e81ed7b06f84b2bab3369334cb1f90a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:57:23.180801 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-c969972e-e11e-4b20-9eee-b4e10a58d48b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:57:23.181433 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-c969972e-e11e-4b20-9eee-b4e10a58d48b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:57:23.183118 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1453/2895] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:57:22 2022] DELETE /identity/v3/projects/8e81ed7b06f84b2bab3369334cb1f90a => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:57:23.186953 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a888a83-7a72-4122-9ba9-ab592d9cbaef admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:57:23.188287 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-9a888a83-7a72-4122-9ba9-ab592d9cbaef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:57:23.201224 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a888a83-7a72-4122-9ba9-ab592d9cbaef admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:57:23.256929 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4348b7d1-c32e-4e58-a534-6acd2bc0749a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:57:23.257550 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4348b7d1-c32e-4e58-a534-6acd2bc0749a admin admin] REQUEST_METHOD: `DELETE` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:57:23.257825 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4348b7d1-c32e-4e58-a534-6acd2bc0749a admin admin] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:57:23.258091 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4348b7d1-c32e-4e58-a534-6acd2bc0749a admin admin] PATH_INFO: `/v3/users/770039941d344dc3a6ea3d7e3d379c78` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:57:23.271168 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4348b7d1-c32e-4e58-a534-6acd2bc0749a admin admin] RBAC: Authorizing `identity:delete_user(user_id=770039941d344dc3a6ea3d7e3d379c78)` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:57:23.273129 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4348b7d1-c32e-4e58-a534-6acd2bc0749a admin admin] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:57:23.316278 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-4348b7d1-c32e-4e58-a534-6acd2bc0749a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '770039941d344dc3a6ea3d7e3d379c78'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:57:23.316908 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-4348b7d1-c32e-4e58-a534-6acd2bc0749a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '770039941d344dc3a6ea3d7e3d379c78'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:57:23.317500 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-4348b7d1-c32e-4e58-a534-6acd2bc0749a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '770039941d344dc3a6ea3d7e3d379c78'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:57:23.324764 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-4348b7d1-c32e-4e58-a534-6acd2bc0749a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '770039941d344dc3a6ea3d7e3d379c78'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:57:23.332090 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.notifications [None req-4348b7d1-c32e-4e58-a534-6acd2bc0749a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '770039941d344dc3a6ea3d7e3d379c78'} {{(pid=85834) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:57:23.349026 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1443/2896] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Aug 17 23:57:23 2022] DELETE /identity/v3/users/770039941d344dc3a6ea3d7e3d379c78 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:57:23.522525 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c969972e-e11e-4b20-9eee-b4e10a58d48b admin admin] Authenticating user token {{(pid=85835) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 23:57:23.524098 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.fernet_utils [None req-c969972e-e11e-4b20-9eee-b4e10a58d48b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85835) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 23:57:23.538182 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c969972e-e11e-4b20-9eee-b4e10a58d48b admin admin] Validating token access rules against request {{(pid=85835) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 23:57:23.578251 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:57:23.596754 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532fd070-4929-4e8b-98f9-3f63e797f770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f1a8fbc56c44c829c8dd8013a782220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd475051e9cd456abbb6cff649b374cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85835) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 23:57:23.597376 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532fd070-4929-4e8b-98f9-3f63e797f770 admin admin] REQUEST_METHOD: `DELETE` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 23:57:23.597651 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532fd070-4929-4e8b-98f9-3f63e797f770 admin admin] SCRIPT_NAME: `/identity` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 23:57:23.597918 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532fd070-4929-4e8b-98f9-3f63e797f770 admin admin] PATH_INFO: `/v3/projects/4184e8c8517b4813b57ba864590cf992` {{(pid=85835) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 23:57:23.607927 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532fd070-4929-4e8b-98f9-3f63e797f770 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4184e8c8517b4813b57ba864590cf992)` {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 23:57:23.609897 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532fd070-4929-4e8b-98f9-3f63e797f770 admin admin] RBAC: Authorization granted {{(pid=85835) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 23:57:23.655294 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-532fd070-4929-4e8b-98f9-3f63e797f770 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4184e8c8517b4813b57ba864590cf992', 'request_id': 'req-532fd070-4929-4e8b-98f9-3f63e797f770'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:57:23.660662 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-532fd070-4929-4e8b-98f9-3f63e797f770 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4184e8c8517b4813b57ba864590cf992', 'request_id': 'req-532fd070-4929-4e8b-98f9-3f63e797f770'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:57:23.667686 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-532fd070-4929-4e8b-98f9-3f63e797f770 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4184e8c8517b4813b57ba864590cf992', 'request_id': 'req-532fd070-4929-4e8b-98f9-3f63e797f770'} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:57:23.679182 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-532fd070-4929-4e8b-98f9-3f63e797f770 admin admin] The token cache is being invalidate because project 4184e8c8517b4813b57ba864590cf992 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=85835) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 23:57:23.679582 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-532fd070-4929-4e8b-98f9-3f63e797f770 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:57:23.680247 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG keystone.notifications [None req-532fd070-4929-4e8b-98f9-3f63e797f770 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=85835) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 23:57:23.681700 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: [pid: 85835|app: 0|req: 1454/2897] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Aug 17 23:57:23 2022] DELETE /identity/v3/projects/4184e8c8517b4813b57ba864590cf992 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 23:57:33.345899 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=77,keystone:DELETE=15,keystone:INSERT=2 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 23:57:33.676329 dv-cinder-820027-20-iscsi devstack@keystone.service[85835]: DEBUG dbcounter [-] [85835] Writing DB stats keystone:SELECT=36,keystone:DELETE=7,keystone:INSERT=1 {{(pid=85835) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 00:02:34.153563 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4348b7d1-c32e-4e58-a534-6acd2bc0749a admin admin] Authenticating user token {{(pid=85834) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 00:02:34.155028 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-4348b7d1-c32e-4e58-a534-6acd2bc0749a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 00:02:34.288845 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4348b7d1-c32e-4e58-a534-6acd2bc0749a admin admin] Validating token access rules against request {{(pid=85834) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 00:02:34.349585 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fcba6c-c236-406f-8fc0-af80a593523e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc57d911a56423e96c736bc96270ba8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de86915ef4340248ba8a8f51d56594e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=85834) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 00:02:34.350182 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fcba6c-c236-406f-8fc0-af80a593523e service placement] REQUEST_METHOD: `GET` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 00:02:34.350458 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fcba6c-c236-406f-8fc0-af80a593523e service placement] SCRIPT_NAME: `/identity` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 00:02:34.350724 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fcba6c-c236-406f-8fc0-af80a593523e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=85834) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 00:02:34.356896 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.fernet_utils [None req-60fcba6c-c236-406f-8fc0-af80a593523e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=85834) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 00:02:34.370581 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fcba6c-c236-406f-8fc0-af80a593523e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 00:02:34.372177 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fcba6c-c236-406f-8fc0-af80a593523e service placement] RBAC: Authorization granted {{(pid=85834) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 00:02:34.381788 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: [pid: 85834|app: 0|req: 1444/2898] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Aug 18 00:02:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 00:02:44.377644 dv-cinder-820027-20-iscsi devstack@keystone.service[85834]: DEBUG dbcounter [-] [85834] Writing DB stats keystone:SELECT=89 {{(pid=85834) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}